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Executive Summary 


School climate refers to dimensions of school life 
(e.g. safety, relationships, teaching and learning, the 
environment) as well as to larger organizational patterns. 
School climate has been extensively studied and has 
been shown to be predictive of academic achievement, 
school success, effective violence prevention, students’ 
healthy development, and teacher retention (Cohen, 
McCabe, Michelli, & Pickeral, 2009). Yet, Cohen et al. 
describe a persistent gap between school climate research 
and policy and practice. In Illinois, the need for the 
measurement of school climate has been addressed by 
Public Act 098-0648“ which requires Illinois schools to 
administer a biennial learning conditions and climate 
survey beginning with the 2012-2013 school year. 
Developed by the University of Chicago Consortium 
on Chicago School Research, the 5Essentials Survey 
of Learning Conditions, hereafter referred to as 
“5Essentials” or “5E” Survey, was developed based 
on nearly two decades of research (Bryk, Sebring, 
Allensworth, Luppescu, & Easton, 20 1 0) and measures 
PK-12 schools’ instructional environment, based 
on teachers’ and students’ input to inform schools’ 
improvement processes in five areas identified as critical 
for school success: Effective Leaders, Collaborative 
Teachers, Involved Families, Supportive Environments, 
and Ambitious Instruction. b Also offered by UChicago 
Impact, was a parent survey supplement, developed by 
Chicago Public Schools to solicit parent feedback on 
school learning conditions and climate. 

The primary purpose of this study was to examine 
how Illinois school districts are utilizing the Illinois 
5Essentials Survey results, particularly for school 
improvement, to determine challenges to successful 
implementation, and to make recommendations for 
improvements to the 5E Survey and implementation 
process for statewide use. We also summarize district/ 
school stakeholders’ familiarity with the 5E Survey, 
perceived value of the survey, preferences for training, 


levels of supports, and reasons for not using the 5E 
Survey data. 

Using the conceptual framework developed by Means, 
Padilla, DeBarger, and Bakia (2009) for data-informed 
decision making, we used a mixed-methods approach 
to determine the supports and challenges for using the 
5E data for decision-making. Fifteen Illinois school 
districts were selected as study sites for further in- 
depth investigation based on several selection factors, 
including early data use, Race to the Top and School 
Improvement Grant status, geographic diversity, 
urbanicity, and enrollment size. Interviews with 79 
district/school administrators and school personnel 
involved in school improvement planning (SIP) 
teams during May-June, 2014, as well as document 
review of the participating districts’ Rising Star school 
improvement plans were key sources of data from the 
study sites. This information was supplemented with 
data from a statewide survey of district/school leaders 
conducted in early 2014 by the Illinois State Board 
of Education (ISBE) that solicited their experiences 
and opinions of the 5Essentials Survey and reports. 
Survey respondents included 273 superintendents (32% 
response rate) and 634 principals (16% response rate). c 

Familiarity & Training 

Superintendents were generally the most familiar 
with the 5E Survey, while school improvement team 
members (typically teachers) were least familiar. This is 
understandable because superintendents and principals 
were the primary contact for the 5E administration; 
however, it may also indicate a lack of information 
dissemination beyond the district or school office. We 
also observed that familiarity with the 5E Survey tended 
to cluster within district, such that in some districts 
familiarity was high across district and school personnel 
while in other districts familiarity with the 5E Survey 
was generally low across district and school personnel. 


1 http://www.ilga.gov/legisktion/publicacts/98/098-0648.htm 

b For more information about the 5Essentials Survey see: https://illinois.5-essentials.org/20l4/ or http://www.isbe.net/5essentials/pdf/2013-l4/faq-01 14.pdf 
c Response rates were estimated based on data from the ISBE Directory of Education Entities for 2013-14 which reported 865 districts and 4008 schools. 
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Participants reported limited training on how to use 
the 5E reports and data. Training typically entailed 
superintendents and district office administrators 
participating in a webinar or short presentation on 
the basics of the 5E Survey and administration issues. 
Principals reported receiving most of their information 
from their superintendents (or district office), with just a 
few receiving additional information through a webinar. 
Some district office personnel and principals sought 
additional information on their own. Most of the 
district and building SIP members were only familiar 
with the 5E Survey because they took the survey in their 
roles as teachers. Some respondents indicated that with 
additional training, the data would be more effectively 
utilized. One respondent commented: 

“I haven’t had a full day of training on 5Essentials 
yet, and much less the kind of training that we got for 
the capacity builders training, where they’re training 
you how to train other people on it. We need that and 
we need that done really effectively at the state level, 
because again, I see a lot of value in this data. I know 
that it won’t get used well unless the districts have 
the ability to handle it themselves and understand it 
enough to support it, so we wouldn’t be using Rising 
Star in our district as much if we weren’t trained as 
capacity coaches. ” 


Level of Supports 

Participating districts were categorized by their level 
of supports for making data-informed decisions based 
on Means et al.’s (2009) six categories, which are listed 
in Table I. With only three exceptions, the overall 
assessments indicated that all of the districts in our 
sample were at least approaching a modest level of 
supports for data-informed decision making. Elowever, 
only one district was rated as having supports available 
to a great extent (see far right column). In the words of 
one of the superintendents, all of these districts are “in 
transition to data-driven decision making.” 

Principals and teachers generally reported having 
access to all or most data for students in their schools, 
or at least for the students in their classrooms. The 
lowest rated support was the provision of professional 
development and technical support for data 
interpretation. Even those schools that have numeracy 
and/or literacy coaches to provide this support expressed 
concern that these positions are grant dependent and 
limited in what they could provide to teachers outside 
of the grant parameters. An exemplar of providing 
data support was in one setting in which district office 
personnel with expertise to serve as data coaches worked 
with teachers to understand the 5E data. 

Unfortunately, data supports did not predictably 
translate into high usage of the 5E data among these 


Table I. Level of Data Supports for Participating Districts 


District 

Locale 

Size 

Type 

Data 

Systems 

Access 

Leadership 

for 

Improvement 

Tools for 
Generating 
Data 

Social 

Structures 

Professional 
Development 
& Tech 
Support 

Tools for 
Acting on 
Data 

District 

Score 

1 

City, Mid 

L 

P-12 

M 

M 

S 

G 

M 

M 

M 

2 

City, Small 

L 

P-8 

G 

G 

G 

G 

G 

G 

G 

3 

City, Small 

L 

P-12 

G 

M 

G 

G 

S 

S 

M+ 

4 

Suburban, Large 

L 

9-12 

M 

M 

S 

M 

S 

-M 

-M 

5 

Suburban, Large 

L 

P-12 

G 

M+ 

G 

M+ 

M+ 

M 

M+ 

6 

Suburban, Large 

L 

P-12 

G 

S 

S 

M 

M 

-M 

-M 

7 

Suburban, Large 

L 

P-8 

G 

M 

M 

S 

S 

M 

M 

8 

Town, Fringe 

M 

P-12 

G 

-M 

M 

S 

S 

S 

S+ 

9 

Town, Distant 

M 

9-12 

G 

M+ 

S 

M 

-M 

S 

-M 

10 

Town, Remote 

M 

P-8 

G 

M 

— 

S 

-M 

M+ 

M 

11 

Town, Remote 

L 

P-12 

G 

M 

M 

S 

S 

S 

-M 

12 

Rural, Fringe 

L 

P-12 

G 

S 

M 

s 

-M 

M 

-M 

13 

Rural, Fringe 

S 

P-12 

G 

M 

M 

G 

-M 

M 

M 

14 

Rural, Distant 

S 

P-12 

M 

S 

S 

S 

S 

S 

S 

15 

Rural, Distant 

s 

K-8 

M 

S 

S 

-S 

-S 

S 

S 


Notes: This data is arranged based on the locale code assigned to the school district by the National Center for Education Statistics. 

Codes indicate the extent to which each support was present during the 2013-2014 school year based on interviews with district/school personnel: 

-S = not at all to some extent; S = to some extent; S+ = to more than some extent; 

-M = to a less than modest extent; M = to a modest extent; M+ = to a more than modest extent; 

G = to a great extent. 

An extended scale is used to report these findings because respondents frequently expressed gradations beyond the four choices (Great Extent, Modest Extent, 
Some Extent, or Not at All) given to them regarding the extent to which their district was moving towards implementation of a data-informed decision making culture. 
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participants. In the 1 5 participating districts, we found 
districts with modest to great supports on both ends of 
the continuum for utilizing the 5E data. Three of the 
districts would be considered advanced 5E data users; 
they shared, reviewed, analyzed, and implemented 
new programs based on their 5E results and they had 
modest levels of supports for data-informed decision 
making. On the other hand, the three districts who 
reported limited use (i.e., shared 5E data only) had 
modest to great levels of data supports. We found only 
a slight relationship between levels of data support and 
use of the 5E data, indicating other factors are also 
contributing to districts’ decisions to utilize the 5E data. 

Value of the 5E Data and Reports 

Many participants indicated the 5E Surveys were helpful 
because they provided data from multiple perspectives 
(i.e., teachers, students, and parents) that they did not 
currently have. Participants indicated the measure scores 
could be helpful, however, many participants were also 
interested in having access to results for the individual 
survey questions. Although a few districts felt the 5E 
data were not at all useful due to their strong concerns 
about the validity of the data, participants from 
several districts said the 5E data were very helpful for 
informing improvement plans, despite data concerns. 


From the survey data, we discovered that less than 
50% of both superintendents and principals believed 
that the teacher surveys, the student surveys, and the 
reports were valuable. Superintendents were significantly 
more negative in their ratings of the value of the 
surveys and reports than principals. In further analyses, 
we determined that there were moderate to strong 
correlations between perceived value of the surveys 
and reports with reported use of the data for district/ 
school planning purposes. There were also moderate to 
high correlations between perceived local interest of the 
survey items with reported use of the data for district/ 
school planning purposes (see Table II). These results 
indicate that those districts/schools that highly value the 
survey data and reports and those that perceive the items 
to be of interest to their district/school are those that 
also have higher 5E data use. 

Use of the 5E Data and Reports 

Use of the 5E data varied greatly among the 
participating districts, ranging from fairly extensive use 
to no use at all. The majority of the districts, however, 
fell somewhere in the middle, using the 5E data for 
a limited number of activities. As shown in Table III, 
most of the superintendents from the participating 
districts reviewed the 5E data results with their 


Table II. Survey Items with Moderate to High Correlations with Use of 5E Data for Planning Purposes 



Continuous 

Improvement 

Modified School 
Improvement Plan 


Superintendents 

Principals 

Superintendents 

Principals 

Teacher 

Survey 

Teacher survey generated data 1 found valuable 

0.52 

0.48 

0.33 

0.41 

Most of the survey questions adequately address 
issues of interest to my district/school 

0.36 

0.34 

0.21 

0.28 

Student 

Survey 

Student survey generated data 1 found valuable 

0.42 

0.42 

0.36 

0.32 

Most of the survey questions adequately address 
issues of interest to my district/school 

0.31 

0.28 

0.28 

0.29 

Reports 

Survey reports provided valuable information in 
my role as superintendent/principal 

0.48 

0.51 

0.45 

0.37 

Survey reports provided new information in my 
role as superintendent/principal 

0.34 

0.44 

0.35 

0.36 


Survey reports provided easy to use information 
in my role as superintendent/principal 

0.51 

0.44 

0.40 

0.36 


Teachers in my district/school found the survey 
reports valuable 

0.51 

0.45 

0.43 

0.43 


Parents in my district/school found the survey 
reports valuable 

0.35 

0.41 

0.36 

0.39 


Large Effect Medium Effect Small Effect 
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principals, with many also reviewing the data with their 
SIP teams or their entire teaching staff. The survey data 
corroborates these findings, as the highest reported uses 
of the data by superintendents were to review/ discuss 
results at the district leadership level (82%) and make 
efforts to ensure that teachers have received the data 
(68%). A high percentage of principals also reported 
making efforts to ensure teachers received the data 
(80%) and reviewing/ discussing the data at the district 
leadership level (70%). Also, another highly endorsed 
use by principals only was reviewing/discussing data 
with the school improvement teams (69%). 

Survey respondents did not report high use of the 5E 
data for planning purposes, 33% of superintendents 
and 48% of principals reported using the data for 
continuous improvement planning, while only 14% of 
superintendents and 28% of principals reported using 
the data for modifying district/school improvement 
plans. However, there were consistent positive moderate 
to high correlations between opinions of the 5E 
reports and use of the 5E data for planning purposes 
(see Table II). Moreover, there were some examples of 
strong integration of the 5E data in a few of the study 
sites. Following the 5E research that indicates strength 
in three of the five Essentials should have a positive 
impact on student outcomes (Bryk et ah, 2010), one 
of the study site districts selected three Essentials as 
their focal points and conducted a SWOT analysis in 
2013, examining Strengths, Weaknesses, Opportunities 
for improvement, and Threats to those opportunities. 
The teachers identified several areas that they felt were 
weaknesses or opportunities for improvement, and 
threats to those opportunities. The school repeated the 
SWOT analysis for 2014 for the same three Essentials, 
and the principal described the 5E data as “informing 
our work for the next year. So everything that we do is 
kind of around the 5Essentials Survey.” 

In our document review of the Rising Star school 
improvement plans, five of the participating 1 3 districts 
that had a SIP in the Rising Star system mentioned 
the 5E data in their plans. Although the level of 
integration of the 5E data was limited in the 1 5 districts 
participating in this study, some districts have begun to 
utilize the 5E data in their school improvement plans. 
One of the study site districts utilized the 5Essentials 
data by integrating it with their Rising Star and teacher 


evaluation data, specifically, in order to set goals for 
school and teacher improvement. Other examples of 
integration with the Rising Star system are provided in 
the full report. 

Impediments to Use of the 5E Data and Reports 

Most of the participants expressed some concerns about 
the 5E data. The main concerns were credibility of 
data, utilization of alternate climate surveys, and being 
overwhelmed by other mandated activities. Many of 
these districts still utilized the data, working within their 
perceived constraints, while other districts simply did 
not utilize the data due to their strong concerns. The 
most cited reason for concern about the credibility of 
the data was the potential for multiple responses from 
the same person or a person completing the survey for 
a school other than their own. Another major concern 
regarding the credibility was perceived problems with 
the survey language, such as confusing wording or lack 
of a “does not apply” option. Other concerns about the 
credibility of the data included poor generalizability 
due to low response rates or the original development 
of the survey for a different context, i.e., Chicago Public 
Schools; implementation issues due to technology 
problems or unclear instructions or definitions; and 
having data only from 6th grade students for K-6 
schools. 

Most of the districts interviewed already administered 
a locally-developed or national climate survey which 
they believed met their needs. This was a concern 
both because they felt the redundancy with the 5E 
Survey was not needed and also because they were 
concerned about the impact of “survey burnout” on 
both the 5E and other surveys’ response rates. Study 
site participants were frustrated by the number of 
new mandated activities (e.g., new administrator and 
teacher evaluation, PARCC, Common Core) and in 
many instances, saw the 5E Survey as one more burden, 
particularly during the spring term, when testing is 
heavy. With unfortunate timing in several districts we 
visited, the 2014 5E Survey administration coincided 
with reduction-in-force notices which could have 
affected opinions on the survey, as well as response rates. 
Other reported impediments to implementation and 
data use included lack of encouragement to use the data 
from school administrators, confidentiality concerns 
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Table III. Summary of Use of the 5Essentials Data by Districts/Schools 


District # 

Reviewed data 

Shared with 
others 

Compared 
5E with 
Rising Star 

Compared 
5E with 
Danielson 

Developed 

goals 

Benchmark/ 
waiting for 
trend data 

Identified 

improvement 

areas 

Identified 

strength 

areas 

Identified 
PD areas 

Used 
with data 
coaches 

Evidence 
for principal 
evals 

Posted on 
website 

4 

X 

with principals 
with SIP team 

School Board 

X 

X 

X 

X 

X 


X 




1 

X 

with principals 
with teachers 
(in detail) 

School Board 
Parents 
Community 




X 

X 

(e.g., increased 
teacher 
observations) 

X 


X 

X 

Possibility 
for future 
to provide 
evidence 

X 

9 

X 

with principals 
with teachers 
with Rising Star 
team 

with Bldg SIP 
team 

School Board 
Parents 
Media 

X 



X 

X 

(e.g., created 
freshman 
monitoring 
program) 

X 





6 

X 

with principals 
with SIP team 

School Board 
Parents 
Community 



X 

X 

X 

X 





8 

X 

with principals 
with SIP team 

School Board 
Parents 
Community 




X 

X 

X 





11 

X 

with District SIP 
Shared with 
principals 

School Board 
Parents 
Community 

X 




X 

X 




X 

10 

X 

with principals 
with teachers 

unknown 





X 


X 




5 

X 

with principals 

Parents 



X 


X 

(principal plans 
for upcoming 
year) 

X 





14 

X 

with teachers 

School Board 



X 

limited 


X 

(future when 
data are valid) 






13 

X 

with principals 

unknown 





X 

(e.g., added 
parents 
to school 
committees) 






3 

X 

with principals 

School Board 

X 

(future 

plans) 



X 

(future 

plans) 







15 

X 

with teachers 

School Board 




X 

(might 
consider 
in the 
future) 

X 

(might consider 
in the future) 






2 

X 

with principals 

unknown 











7 

X 

No 











12 

X 

superintendent 

only 

No 












NOTE: Information gathered from interviews with 15 districts. 
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if™ 

in small districts, lack of information about the survey 
and its technical properties, concerns about its use for 
evaluative purposes, concerns about the insufficient 
financial support for implementing the survey, and 
public posting of the data, especially in the context of 
the concerns about data credibility. 

Suggestions for Improvement 

Participants had numerous suggestions for improving 
the 5E Survey and its implementation. 

1. Market the survey — frame and motivate 
participation and utilization — in order to increase 
buy-in 

Based on both the comments regarding the value 
of the 5E and the moderate to strong positive 
correlations of survey value and use, if perceived 
value can be increased, one would expect more buy- 
in and better use across and within districts. In the 
words of one respondent: “ Provide us something that 
meets a need and we’re probably all ears. ” 

2. Increase breadth of participation and response 
rates 

Many suggestions were made regarding how to 
increase participation rates, including specific 
strategies for higher parent participation, emphasizing 
the importance of the survey, and letting respondents 
know that results will be publicly posted. 

3. Rostering 

To address the issue of potentially invalid entries, 
many participants suggested using a rostered survey 
to ensure that participants do not take the survey 
more than once or for the wrong school. They 
noted that rostering would also provide survey 
administrators with the ability to monitor responses 
in order to target reminder notices and check the 
representativeness of local responses. 

4. Improving the survey items 

Several suggestions were made to revise items to 
reflect the specific school context and to clear up 
confusing wording and make the survey more parent- 
friendly. Additional suggestions included adding a 


progress bar to the online survey, providing login 
assistance, and simplifying the login process. Many 
respondents suggested making the survey shorter 
to help ensure respondents are taking the survey 
seriously and to increase response rates. 

5. Timing of the survey 

For reasons previously noted, respondents felt the 
current survey administration window should be 
moved earlier in the school year — but not so early 
that respondents cannot accurately respond to items. 

6. Provide more actionable feedback 

Recommendations for improving the presentation 
and usefulness of the survey results also emerged 
from the study. In order to increase utilization of the 
5E results, many respondents expressed the need for 
training and explicit directions for using the data. 
Related suggestions included allowing for time to be 
set aside to review and interpret the survey results 
as a school-wide or district-wide team. Importantly, 
several participants also noted that they needed 
assistance implementing actionable plans. One 
participant noted: 

“But we’re at the point now where we need help with 
the implementation, having to help principals. How do 
I use this information to put into my plans ? What does 
going from yellow to red and green look like in terms of 
a lesson plan l You know, boiling it down to that level. ” 

Progress to Date 

It should be noted that ISBE and UChicago Impact 
have begun to address many of the concerns raised 
by interview participants and survey respondents. For 
instance, in Spring 2014, they conducted focus groups 
with administrators and teachers around the state to 
solicit feedback for problematic language or context 
and developed a set of recommendations. 11 In addition, 
pilot testing of alternative language for survey items 
took place during the 2014 survey administration and 
a pilot study utilizing rosters for survey participation 
was conducted in Spring 2014 to examine issues with 
multiple entries and invalid respondents. e Based on 
the roster pilot study, ISBE, in consultation with 


d http://help.ccsrsurvey.uchicago.edu/customer/portal/articles/l461 l43-illinois-5essentials-focus-group-report 
e For more info on roster pilot study see: http://www.isbe.net/5essentials/pdf/5E-roster-survey-rptl4.pdf 
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UChicago Impact, decided to roster all student surveys 
and extend the rostering pilot for the teacher survey 
to 75 districts for the 2015 survey administration. 
Moreover, the 2015 5E administration dates have been 
moved earlier in the spring term to January 12, 2015 - 
March 13, 2015. Further, the 2013 and 2014 data have 
been re-benchmarked to 2013 data for similar Illinois 
schools. Additionally, Public Act 098-0648“ was recently 
modified to allow school districts to elect to use an 
alternate survey which is required to be pre-approved by 
ISBE. 

Conclusions and Implications 

Some stakeholders found value in the 5E data because 
it added additional perspectives, although this was not 
true for the majority of the participants. It is interesting 
to note that principals generally had a more positive 
view of both the value of the 5E Surveys and reports 
and their local relevance than superintendents did. We 
also found that use of the 5Essentials results is still in 
early stages of adoption for many districts in which 
usage was primarily limited to general discussions about 
districts’ strengths and weaknesses. Additionally, many 
participants, particularly teachers on SIP teams, were 
not familiar with even the basic information about 
the 5Essentials and research supporting its use. The 
findings indicate that there is room to grow in building 
consensus around the value of the 5E data and this 
would likely facilitate more use of the survey data and 
reports. Given the between-district differences in data 
use, this may be most effective if district leadership were 
convinced of the relevance and value of the data and 
reports. 

Respondents noted many reasons for not utilizing the 
5E Survey results. Concerns about the credibility of the 
data created major barriers for many districts to utilize 
the 5E results and to generate buy-in. Many districts 
had their own climate survey that they perceived as 
providing comparable information and they did not see 
the need for an additional survey due to both time and 
cost pressures. The 5E Survey implementation was one 
of many recent new initiatives in the state, increasing 
the perception of the 5E Survey as a burden or an 
unfunded mandate. Despite these concerns, there were 


districts which perceived some usefulness of the data 
and were using it in some capacity. 

The participants in this study had several ideas 
for improvement, both for the survey and for 
implementation and use of the data and reports. Some 
of the frequently noted ideas included increasing 
training, similar to what is provided for the Rising Star 
system; inspiring the district leadership to create a data 
culture, receptive to the value and possibilities of the 
data; and providing additional supports, especially for 
districts with fewer financial and personnel resources. 

In addition, we determined that there are some districts 
with more buy-in and/or resources, in which the 5E 
data are being used in innovative and seemingly effective 
ways. Creating a learning community of district/ 
school data users to share workable and successful data 
integration practices would be one strategy to scale up 
the effective practices that are already occurring around 
5E data and report use. 

To conclude, utilization of the 5Essentials Survey of 
Learning Conditions results varied widely across the 
1 5 participating districts in this study, ranging from 
districts that have implemented programs to districts 
refusing to use the 5E data due to data validity issues 
and other concerns. Steps to address some of these 
issues are currently underway, which will likely increase 
buy-in in the future. Additional strategies such as 
training or external resources to increase districts’ 
capacity to analyze and apply the 5E results to school 
improvement planning may be needed in districts with 
fewer data supports and emerging data-based cultures. 

A positive environment in which districts are interested 
in collecting school culture and climate data from their 
stakeholders to improve their schools currently exists 
in Illinois. The ability to capitalize on and cultivate 
these positive attitudes to increase utilization of the 
5Essentials data, or other climate survey data, will be 
greatly influenced by addressing the concerns identified 
by its stakeholders. 
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Introduction 


In 2012-13, the Illinois State Board of Education (ISBE) implemented the 5Essentials Survey, a 
learning conditions and climate measure, with the goal of providing useful data to guide school 
improvement planning. The 5Essentials Survey fulfills the requirement of Public Act 098-0648 
(http://www.ilga.gov/legislation/publicacts/98/098-0648.htm) to administer a biennial statewide 
learning conditions and climate survey starting in the 2012-2013 school year. Developed by 
the University of Chicago Consortium on Chicago School Research (CCSR), the 5Essentials 
framework was developed based on prior evidence of effective schools, extensive experience in 
school settings, and continuous conversations with education practitioners and other stakeholders, 
and close to two decades of research in the Chicago Public Schools (Bryk, Sebring, Allensworth, 
Luppescu, and Easton, 2010). The 5Essentials Survey measures PK-12 schools’ instructional 
environment, based on teachers’ and students’ input to inform schools’ improvement processes in 
five areas identified as critical for school success: Effective Leaders, Collaborative Teachers, Involved 
Families, Supportive Environments, and Ambitious Instruction. 1 Also offered by UChicago Impact 
was a parent survey supplement, developed by Chicago Public Schools, to solicit parent feedback 
on school learning and climate. 

Data-informed decision making begins with the availability of data but also requires a data culture 
involving expectations and practices with a continuous improvement perspective to determine the 
effectiveness of education activities to improve outcomes for students (Means, Padilla, DeBarger, & 
Bakia, 2009). This research study explores the second step in this description, that is, how Illinois 
school districts are using the Illinois 5Essentials (5E) Survey data for school improvement. The 
goal of this project is to examine districts’ implementation of the survey and the utilization of its 
results. This study is one of several projects examining the relevance and predictive validity of the 
5E Survey for statewide use. 


This study explores 
how Illinois schools 
are using the Illinois 
5Essentials Survey 
data for school 
improvement. 


To examine how schools are utilizing the 5Essentials Survey data, we investigated Illinois school 
districts’ familiarity with the 5E Survey, use of the 5E data for school improvement, reasons for not 
utilizing the 5E data, and suggestions for improving the 5E Survey and its implementation. We 
also gauged districts’ level of data supports available to guide data-informed decisions, for both the 
5E data as well as other data utilized by their district, using the conceptual framework developed 
by Means et al. (2009), in which they identified six major types of prerequisites and supports for 
data-informed decision making. We examined the extent to which these supports were present at 
1 5 selected school districts and the relationships between the level of data supports and districts’ 
utilization of the 5E data. Lastly, we analyzed how the 5Essentials Survey data is being used 
statewide based on an ISBE Implementation Feedback Survey conducted in January and February 
2014 with Illinois public school principals and superintendents. 

The intent of this study was not to evaluate districts’ use of the 5E data, but rather to provide an 
inventory of current practices around the 5E data statewide and to provide recommendations to 
ISBE to increase utilization of the 5E Survey results. Additionally, specific examples from districts 
utilizing the 5E data are included to inform other districts of possible options for utilizing the 5E 
data. While results from the qualitative data analyses cannot be generalized to all Illinois public 
school districts, they are intended to inform stakeholders about potential next steps in their 
activities. 

For more information about the 5Essentials Survey see: https://illinois.5-essentials.org/20l4/ or 
http://www.isbe.net/5essentials/ pdf/20 13-14/ faq-0 1 l4.pdf 
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The study employed a mixed-methods approach, using interviews with administrators 
and teachers from 1 5 purposefully selected Illinois school districts, document review of 
districts’ Rising Star school improvement plans, and quantitative and qualitative data from 
ISBE’s online survey in January 2014 with Illinois superintendents and principals regarding 
their experiences with and opinions of several Illinois education initiatives, including the 
5Essentials Survey. For the interviews and the Rising Star documentation review, districts 
were selected based on ISBE recommendations, and Race to the Top (RttT) and School 
Improvement Grant (SIG) status, ensuring representation from different regions of the 
state, different levels of urbanicity, and different district enrollment sizes. We recruited both 
early users and non-users of the 5Essentials Survey data, based on their responses to 5E 
data use questions from the January 2014 ISBE survey on education initiatives. Chicago 
Public Schools was intentionally not selected because of the long history of use of the 5E 
Survey. All selected sites were willing to participate despite the timing of the request at the 
end of their school year. Because participants were not randomly selected or necessarily 
representative of schools throughout the state, the results of this should not be generalized 
to schools outside of this sample. Table 1 provides a breakdown of the districts by selection 
criteria. 

Table 1 . Description of Participating Districts for Interviews 



Number of 


Number of 


Districts 


Districts 

Size of District 

Locale 

Large (> 2000) 

9 

city - mid 

1 

Medium (600- 1200) 

3 

city - small 

2 

Small (< 600) 

3 

suburban - large 

4 

District Type 

town - fringe 

1 

P-12 

9 

town - distant 

1 

P-8 

3 

town - remote 

2 

9-12 

2 

rural - fringe 

2 

K-8 

1 

rural - distant 

2 

Region 



Northeast 

6 

Race to the Top 

3 

Northwest 

1 



East Central 

3 

SIG 

3 

West Central 

1 



Southeast 

2 



Southwest 

2 




Superintendents from the selected school districts were contacted by email or telephone 
to request their districts’ participation in the research study. Superintendents (or their 
designees) identified key district personnel and a building principal to participate in the 
interviews. Principals then identified key personnel within their building. We requested 
to meet with individuals who were familiar with the 5E Survey or who were responsible for 
school improvement planning. 



The interview protocol (see Appendix A) included questions about the participant’s 
familiarity with the 5E Survey and data, their attendance at and preferences for training 
on the 5E Survey, and their district/school improvement planning processes. Participants 
were also asked whether they used, planned to use, or did not use the data results. For 
those districts that used or planned to use the 5E data, examples of activities utilizing the 
5E data were solicited, along with perceptions of usefulness/value of the 5E data. For those 
not using the 5E data, reasons for this choice were requested. All participants were asked 
about challenges and barriers to utilizing the 5E Survey and data, as well as suggestions for 
improvements. 

To gauge districts’ levels of data supports available to guide data-informed decisions, for 
both the 5E data as well as other data utilized by their district, we also included questions 
based on the conceptual framework developed by Means et al. (2009) for the national Study 
of Education Data Systems and Decision Making, sponsored by the U.S. Department of 
Education’s Policy and Program Studies Service. Means et al. identified six major types of 
prerequisites and supports for data-informed decision making: 

(a) state, district and school data systems; (b) leadership for educational 
improvement and the use of data; (c) tools for generating actionable data; (d) social 
structures and time set aside for analyzing and interpreting data; (e) professional 
development and technical support for data interpretation; and (f) tools for acting 
on data (p. 3). 

We modified and eliminated some of the questions due to interview time constraints. 

Interviews were conducted with 79 individuals from 15 Illinois public school districts from 
May-June 2014. Interviews lasted approximately 45-60 minutes and were audio-recorded 
and transcribed to ensure accuracy. Six researchers with experience conducting interviews 
were trained to use the interview protocol and were provided background information on 
the districts for which they conducted the site visits. Interviewer notes were taken for data 
analyses. Participants were allowed to skip any questions and were assured confidentiality. 
Site visits with face-to-face interviews were conducted at 13 districts and telephone 
interviews were conducted with two districts to accommodate their availability. The number 
of individuals interviewed at each site ranged from 2-10, primarily due to the size of the 
district. Some interviews were one-on-one while others included 2-5 individuals as a group. 
Three site visits did not include school improvement team members due to interviews 
occurring after the school year ended. At all site visits, the superintendent (or a designee) 
was interviewed. The building principal was interviewed, at all but one site. Additional 
interviewees included other district administrators, assistant principals, district and school 
improvement planning (SIP) team members (these were typically teachers), and Rising Star 
process managers. 

Audio recordings were transcribed and reviewed independently by two researchers to 
develop a coding scheme. The transcripts were coded by two researchers using Atlas. ti 
qualitative software and reviewed for inconsistencies by a senior researcher. Data saturation 
was achieved, meaning that all of the qualitative responses could be categorized and all 
codes from the interview data were represented. Overarching themes from the codes were 
developed and similarities and differences between district size and locale were examined. 



Means et al. ’s 
framework was used 
to gauge districts’ 
levels of data support: 
data systems, 
leadership, tools 
to generated data, 
social structures, 
professional 
development and 
technical support, 
and tools for 
acting on data. 


w 


M 




Seventy-nine 
educators from 
1 5 Illinois public 
schools participated 
in interviews. 
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Use of the Illinois 5Essentials Survey Data 



Districts’ Rising Star school improvement plans (where available) were reviewed for 
references to 5E Survey data utilization. 


Superintendents’ and 
principals’ responses 
to a statewide 
online survey about 
their opinions of 
and experiences 
with the 5E Survey 
were analyzed. 


In addition to the interviews and document review for the 1 5 selected districts, responses 
to questions pertaining to the 5Essentials Survey from an online survey conducted by 
ISBE in January-February 2014 were analyzed. Superintendents and principals from all 
Illinois public schools were surveyed about their opinions of and experiences with several 
ISBE initiatives, including the 5E Survey. Survey respondents, hereafter referred to as 
“respondents,” were asked to indicate their level of agreement on a four-point scale (Strongly 
Agree to Strongly Disagree) for questions concerning the 5E teacher survey, 5E student 
survey, 5E survey reports, and use of the 5E data. Demographic information for each 
participant’s district or school was also collected, including region of the state, enrollment, 
Race to the Top participation, School Improvement Grant status, and implementation of the 
5E Survey prior to Spring 2013. Survey data were analyzed with descriptive and inferential 
statistics using percentages, cross-tabulations, and chi-square analyses. Familywise error 
rate was corrected using Benjamini and Hochberg’s (1995) false discovery rate to balance 
concerns of inflated Type I error when multiple tests are conducted and to allow maximum 
exploration of the data. A separate correction was conducted for each subscale of the survey 
items. Favorable and unfavorable opinions were determined by aggregating strongly disagree 
and disagree responses and aggregating the total of the strongly agree and agree responses. 
The response patterns of principals and superintendents were also compared, and differences 
by region and district/school size are noted, where significant. Respondents were also given 
the opportunity to include comments about the 5E Survey, and the qualitative data were 
reviewed for emerging themes using Miles and Huberman’s (1994) checklist matrix method. 
Similarities and differences in responses by district size and locale were noted for the 
interview data. 
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Results 


Familiarity with the 5Essentials Survey 

We asked participants from the 1 5 selected districts how familiar they were with the 5E 
Survey and the data from their district or school. Participants reported a broad range of 
familiarity with the 5E Survey, and we classified responses into three categories — Not 
Familiar, Somewhat Familiar, and Very Familiar. Responses were coded “Not Familiar” if 
the participant knew little about the survey beyond the requirements to complete it or if the 
participant had not viewed the survey results for their district/school. Just under one-third 
of participants (approximately 29%) were coded as “Not Familiar,” and many in this group 
were from school improvement teams. Responses were coded as “Somewhat Familiar” if the 
participant was familiar with some aspects of the survey but not others, or if they had viewed 
the results for their district/school but not used them or discussed these with colleagues. 
Participants were coded “Very Familiar” if the they were well acquainted with their school’s 
or district’s survey results and had used them or discussed them with colleagues. These two 
groups were fairly equal with just over one-third of participants in each group, with slightly 
more participants coded as “Somewhat Familiar” with the 5Essentials survey. 

There were a few noteworthy trends with regard to familiarity with the 5Essentials. First, 
school improvement team representatives (typically teachers) tended to be least familiar with 
the survey (beyond just taking the survey), while the superintendents we interviewed were 
generally the most familiar. This could indicate that knowledge of the survey diminishes as 
it is transferred from the district and school offices out to teachers and other school building 
personnel. Second, we noted that familiarity with the survey tended to cluster within district. 
That is, we observed some districts where familiarity was generally high across the board 
and other districts where familiarity was generally low across the board. We also found 
that participants tended to be much more familiar with the processes associated with the 
5Essentials (local response rates, participants, survey items, etc.) than they were with the 
results for their particular school or district. This could indicate that information shared with 
was more about the compliance aspects of survey administration than about the substantive 
feedback on school learning conditions contained in the survey results. 

Participation in and Preferences for Training 

Interview participants were asked if they received any training about the 5Essentials 
Survey or the data. Nearly all of the superintendents and district office administrative staff- 
indicated they received at least some basic information regarding the 5Essentials Survey, 
typically through a webinar or a presentation at a regional meeting. The content of these 
webinars typically included general background about the 5Essentials Survey and logistics 
for administering the survey (e.g., timeframes, login information, suggested communication 
with teachers and parents). Many found the webinars helpful, while others did not. Some 
participants also sought out additional information on their own through the 5Essentials 
website. One superintendent received information through the district’s Race to the Top 
network at a state-level conference. Another district received training through a partnership 
with University of Chicago, with a local foundation covering the expenses of the additional 


Roughly one-third of 
participants fell into 
“Not, ” “Somewhat, ” 
and “Very” familiar, 
with slightly more 
“Somewhat” and 
“Very” familiar with 
the 5E Survey. 


M 




School improvement 
teams were least 
familiar. Familiarity 
tended to cluster 
within districts. 
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Use of the Illinois 5Essentials Survey Data 



Most superintendents 
and district 
administrative staff 
received basic 
information via 
webinars. Principals 
and SIP members 
were less likely to 
receive training. 


training, in which their district administrators, building principals, teacher representatives, 
and parent representatives participated in the training that explained the rationale of 
the 5E Survey, the implementation, and some interpretation of the results. Only a few 
principals received training via webinars beyond what their superintendents (or other district 
administrators) shared with them through informational documents about the 5E and 
logistics for administering the survey. A couple of districts provided principal workshops 
to disseminate more detailed information, but this was limited due to changes in scoring 
the 5E data. Most of the district and building SIP members were only familiar with the 5E 
Survey because they had taken the survey, and they did not typically receive any additional 
information about the 5E Survey. For example, when asked if s/he had received any training, 
one participant remarked: 

“No, none whatsoever. So we just basically take the survey and that’s the first and last time 

that we pretty much see it. ” 

A couple of individuals did receive minimal information, however, through their roles as 
process managers for Rising Star in their districts. 


Training preferences 
were evenly split 
between webinars and 
face-to-face training. 


We also asked for participants’ preferences for how the training should be delivered. 
Responses were evenly split between webinars and face-to-face meetings. Webinars offered 
convenience in terms of location and timing, and were considered less expensive. Face-to-face 
training provided opportunities to share ideas with others and to tailor training topics (e.g., 
urban versus rural issues). One participant suggested a “train the trainer” approach in which 
a district representative would attend a face-to-face workshop and then bring that knowledge 
back to their district for many others to learn together. Other suggestions included providing 
training through the regional offices of education or at professional association meetings. 


Perceived Usefulness/Value of 5Essentials Data 

We asked participants how useful or valuable they thought the 5E data were in their school 
improvement planning. Many participants indicated it was helpful because it provided data 
from multiple perspectives (teachers, students, and parents) that they did not currently have: 

“I feel that with the 5Essential survey, the benefit is it allows schools like this one to tell a story 
about their data as opposed to just everybody looking at it like ‘oh it’s terrible. ’It allows us to 
say there are good things going on despite the fact that we have some growing to do in terms of 
test scores, and graduation rate. So it provides another lens for people to look through in terms 
of our school. ” 




5E data is another 
useful tool, but it is 
merely one piece 
among many others. 




When asked about usefulness of the separate 5E scale scores (i.e., 5E measure scores), 
participants indicated the scale scores could be helpful, however many participants were 
also interested in continuing to have access to results for the individual survey questions. In 
addition, a few administrators commented that they found the data helpful for assessing their 
leadership skills. Participants from many of the districts described the 5E data as another 
useful tool, but that it was merely one piece of data among many others. Although all of the 
different data were helpful, they could be overwhelming, as illustrated by this remark: 


“What we’re trying to do is align all of our efforts together, because if they’re in isolation, 
everybody just gets overwhelmed and if it’s more initiatives, we can’t be successful. So it’s 
trying to fit the puzzle pieces together where it all looks the same. It’s the same work, it’s just 
measuring it differently. ” 
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Participants from several districts found the 5E data had some value but were not extremely 
positive because of low response rates, lack of longitudinal data, issues with scoring, and 
missing information about the local context that may influence survey responses. Although 
a few districts felt the 5E data were not at all useful due to their strong concerns about the 
validity of the data, participants from several districts said the 5E data were very helpful 
for informing improvement plans, despite data concerns. One participant remarked, “in 
that while all of this is brewing, there are still indicators here that very strongly can support your 
improvement efforts. ” 

Use of 5Essentials Survey Data 

One primary purpose of this study was to learn how selected districts are utilizing the 
5E data, particularly for school improvement planning, so that examples could be shared 
with other districts interested in making use of the data. We asked participants if their 
district or school used or planned to use the 5E data and the extent to which the data were 
useful. Participants were not always clear whether the activity was directly related to school 
improvement planning or just a general activity (e.g., reviewing the 5E data during a school 
staff meeting). It does appear, however, that all of the activities could be directly or indirectly 
related to school improvement. 

Table 2 lists the 1 5 districts with their locale, size, and type. The number of districts is 
referenced throughout the rest of the report, rather than identifying the district. 

We provide this information about use of the 5E data in two different approaches. First, 
Table 2 summarizes participants’ responses about the varying ways they used the 5E data, 
followed by a discussion of the information in the table. Then we provide detailed examples 
of select districts’ use of the 5E data. 

Based on the interview data, we found that use of the 5E data varied greatly among the 
participating districts, ranging from fairly extensive use to no use at all. The majority of the 
districts, however, fell somewhere in the middle, in that they have used the 5E data for some 
activities but not for others. 


“But I think it was helpful 
for us and it promoted good 
articulation throughout the 
building with teachers at 
different grade levels. And 
it always helps us with our 
school improvement plan and 
just giving us information 
where we need to move 
forward and the goals we 
need to set for the next year. ” 
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Table 2. Summary of Use of the 5Essential Data by Districts/Schools 


District # 

Reviewed data 

Shared with 
others 

Compared 
5E with 
Rising Star 

Compared 
5E with 
Danielson 

Developed 

goals 

Benchmark/ 
waiting for 
trend data 

Identified 

improvement 

areas 

Identified 

strength 

areas 

Identified 
PD areas 

Used 
with data 
coaches 

4 

X 

with principals 
with SIP team 

School Board 

X 

X 

X 

X 

X 


X 


1 

X 

with principals 
with teachers 
(in detail) 

School Board 
Parents 
Community 




X 

X 

(e.g., increased 
teacher 
observations) 

X 


X 

9 

X 

with principals 
with teachers 
with Rising Star 
team 

with Bldg SIP 
team 

School Board 
Parents 
Media 

X 



X 

X 

(e.g., created 
freshman 
monitoring 
program) 

X 



6 

X 

with principals 
with SIP team 

School Board 
Parents 
Community 



X 

X 

X 

X 



8 

X 

with principals 
with SIP team 

School Board 
Parents 
Community 




X 

X 

X 




X 

with District SIP 
Shared with 
principals 

School Board 









11 

Parents 

Community 

X 




X 

X 



10 

X 

with principals 
with teachers 

unknown 





X 


X 


5 

X 

with principals 

Parents 



X 


X 

(principal plans 
for upcoming 
year) 

X 



14 

X 

with teachers 

School Board 



X 

limited 


X 

(future when 
data are valid) 




13 

X 

with principals 

unknown 





X 

(e.g., added 
parents 
to school 
committees) 




3 

X 

with principals 

School Board 

X 

(future 

plans) 



X 

(future 

plans) 





15 

X 

with teachers 

School Board 




X 

(might 
consider 
in the 
future) 

X 

(might consider 
in the future) 




2 

X 

with principals 

unknown 









7 

X 

No 









12 

X 

superintendent 

only 

No 










NOTE: Information gathered from interviews with 15 districts. 
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Table 3. Participating Districts 


Evidence 
for principal 
evals 

Posted on 
website 

Other 

Other data available 

X 

Possibility 
for future 
to provide 
evidence 

X 

Added supplemental question for parents 
to tie into local survey from previous years; 
waiting for data to be normed against state 
for SIP use 


Used 5E data in SWOT analysis 


Future plans to implement local school 
climate survey to compare with 5E results 


Other data available; SIP members 
reported not receiving the data 


Other data are available 


Other data available although interest in 
using 5E data 


Used somewhat for evidence for 
assessments; compare student 5E 
responses with target core classes; 
concerns about data validity limited use 
Other data available; did not use due to 
data validity concerns 


Plans to triangulate data with local survey 
on learning standards for school leaders; 
plans to use 5E data to inform current 
school initiatives 

Might use in district technology plan; did 
not use due to data validity concerns; 
teachers reported not receiving data 


Other data are available; did not use due 
to data validity concerns; potential use if 
data are valid to confirm other district data 
Other data available; currently contracted 
with teachers' union to continue with 
current internal survey; did not use 
because it is not required by state 
Other data available; did not use due to 
data validity concerns 


District 

Locale 

Size 

Type 

1 

City, Mid 

L 

P-12 

2 

City, Small 

L 

P-8 

3 

City, Small 

L 

P-12 

4 

Suburban, Large 

L 

9-12 

5 

Suburban, Large 

L 

P-12 

6 

Suburban, Large 

L 

P-12 

7 

Suburban, Large 

L 

P-8 

8 

Town, Fringe 

M 

P-12 

9 

Town, Distant 

M 

9-12 

10 

Town, Remote 

M 

P-8 

11 

Town, Remote 

L 

P-12 

12 

Rural, Fringe 

L 

P-12 

13 

Rural, Fringe 

S 

P-12 

14 

Rural, Distant 

S 

P-12 

15 

Rural, Distant 

S 

K-8 


As shown in the Table 2, most of the superintendents 
from the participating districts reviewed the 5E data 
results with their principals, with many also reviewing 
the data with their SIP teams or their entire teaching 
staff. We heard examples from several districts about 
their detailed review of every survey question with their 
SIP team or their entire school staff to carry out many 
activities related to school improvement. For example: 

“We went through every question and then we looked at 
reasons for why we might be at ‘strongly disagree’. ” 

“I went through the entire data that came out. Everyone 
on our team analyzed it and then we bulleted out what 
we believed to be important significant highs and lows, 
positives or negatives that seemed to come out of the 
data from both the combination of the student data and 
teacher data. ” 

Many of the districts also shared their 5E results with 
their school board, with slightly fewer sharing results with 
parents/community. A couple of districts did not widely 
share the 5E results because of concerns about the validity 
of the 5E data or because the availability of other school 
climate data. For those districts that used the 5E data, 
participants described using the 5E results most often to 
identify areas for improvement. For example: 

“Well, just like we use any of the other surveys when 
we’ve surveyed our parents or our staff or students, you 
want to look at the results and see how they align to 
what we think are strengths and weaknesses and then 
plan accordingly. ” 
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“We actually were enlightened by some of the results of the data that we could then better, as 
a large staff, understand why something came back the way it did or what improvements we 
could make based on that data. ” 

“Well I think it was first made clear to me when I first looked at it in terms of effective 
leadership, that I am not communicating well enough with the teachers, . . . but I need to 
make sure that the teachers recognize the vision at least from me and can share in it with me. ’ 

A few districts took the next step and made changes based on their 5E results, such as adding 
more opportunities for teacher observations, adding parents to school committees, and 
creating a freshman monitoring program (see more description in the next section). Districts 
also commonly used the 5E data to identify areas of strength, using this information to 
celebrate their successes and communicate these to their teachers, parents, and community. 
For example: 

“The 5Essentials can help you take a look at where you have all greens but maybe there are 
some areas where you didn’t know how green it was. You could really tout or highlight [those 
successes.]” 

“We actually also used the data to celebrate progress. ” 

For the districts that use the 5E data, almost half considered using the 5E data in subsequent 
years of data to examine trends. Some of these individuals preferred to wait until multiple 
years of data were available before making any significant changes, rather than reacting to a 
single data point. For example: 

“As we get a second set of data we’re going to compare it to the first set of data. We’re going to 
look at it and decide what jumps out at us as strengths and weaknesses and then well have 
two sets of data to move forward in our discussions. ” 


5E data used: 

• to identify 
weaknesses and 
strengths 

• to examine trends 


to develop goals 

to integrate with 
Rising Star 

to identify 

professional 

development 

to communicate 
schools’ results. 


For several districts, the 5E results were used to develop goals for their district/school’s 
strategic plan or for their indicators for their school improvement plans. For example: 

“I use the 5Essentials at my building. We looked at areas of growth . . .from 5Essentials from 
what our parents were saying and also what the teachers were saying and then we wrote goals 
for our building in terms of what we could improve so they were actual specific building goals 
from that data. ” 


Participants from several districts described school administrators and SIP teams working 
closely to integrate their 5E results with their Rising Star school improvement process. 

For instance, the 5E data were used as evidence for the Rising Star indicators in their school 
improvement plans (see the insert “Document Review of Rising Star School Improvement 
Plans” for more information in shaded box). When asked how districts most often use the 5E 
data for school improvement planning, a participant explained: 


“Through the Rising Star team. We looked at the responses. We felt like the areas that we 
were already working on in our school improvement process, the 5 Essentials data just kind 
of cemented the things we know we need to progress and work on. It really helped us know 
that we were moving in the right direction. I think it compliments what we’re doing in school 
improvement, specifically in our Rising Star team very, very well. ” 
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Document Review of Rising Star School Improvement Plans 

We examined the comprehensive reports available in the Rising Star system for the 15 participating districts to 
identify examples of districts incorporating the 5E data into their school improvement plans. Of the 1 3 districts 
that had a school improvement plan in the ISBE Rising Star system, five districts mentioned the 5E data in 
their activities or plans. The 5E Survey was discussed in both plans and tasks associated with different Rising 
Star indicators. Table 4 presents the Rising Star indicators tied to the 5Essentials. As seen in the table, one 
indicator, Cl 1 1 , was referenced by three out of the five districts, while the other indicators typically had just one 
district incorporating the 5E data with the Rising Star indicator. 

The focus of the indicators was quite varied, however, the tasks associated with the 5E data typically involved 
implementing the 5E Survey and reviewing results. Some activities, such as aligning the 5E data with other 
indicators or using the 5E data for evidence of implementation, demonstrate more integration of the 5E data 
with the district’s other data systems. Although the level of integration of the 5E data is limited with the 15 
districts participating in this study, some districts have begun to utilize the 5E data in their school improvement 
plans. 

Table 4. Summary of Rising Star Indicators Utilizing 5Essential Survey for 15 Participating Districts 


# of districts 
(out of 5) 

Indicator # 

Indicator 

Example of plan or task 

3 

cm 

The district and schools will have an aligned 
vision/mission statement that supports a learn- 
ing environment which is emotionally safe and 
conducive to learning. 

Attend 5E webinars. Develop administration 
plan for 5E. Implement 5E. Review results 
when available. 

2 

CM2 

The district improvement process will be aimed 
at student academic, physical, social, emotional, 
and behavioral development. 

Teachers and students complete 5E Survey. 
SIP will review 5E data and create action plans 
on areas identified for improvement. 

1 

Dll 

The district will ensure the delivery of the cur- 
riculum is differentiated to meet the needs of all 
learners. 

Align Rising Star indicators with data from 5E 
Survey. Danielson Framework, and District’s 
Indicators of Success. 

1 

IA01 

The district will build partnerships with municipal 
and civic leaders, includes them in district and 
school improvement planning, and maintains 
regular communication with them. 

Implement the 5E Survey. 

1 

IB07 

The district will ensure that school improvement 
and restructuring plans include research-based, 
field-proven programs, practices, and models. 

District to participate in the 5E Survey and use 
data to improve school practices and strate- 
gies. 

1 

1 B 1 0 

The district ensures that the change agent 
(typically the principal) is skilled in motivating 
staff and the community, communicating clear 
expectations, and focusing on improved student 
learning. 

5E data, along with other data, to be used as 
evidence that this indicator has been imple- 
mented. 

1 

IC03 

District and school decision makers will meet 
at least twice a month to discuss the school’s 
progress. 

Discuss 5E Survey and results. 

1 

Rt3-1 

The school district’s teacher and principals’ 
evaluation systems will incorporate both profes- 
sional practice and student growth and evalu- 
ation information is used to improve educator 
effectiveness. 

Plans to use 5E Survey data: Academic 
Personalism as evidence this indicator is fully 
implemented. 
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Lastly, one or two districts used their 5E results to identify professional development 
topics for teachers or to communicate their results by posting on the district website. 
Notably, one district which utilized the 5E results extensively used district data coaches to 
assist teachers with interpreting their 5E results and discussed potentially using the 5E data 
as evidence in principal evaluations in the future. For example: 

“ Our assessment specialists often go directly to buildings and work with staff in order to really 
be able to answer questions and to make sure that there’s some integrity and validity and 
uniformity in the understanding, so that people aren’t looking at it from multiple lens and 
maybe, as we look to make district decisions, we’re not all over the place. So our experts really 
do help us make that happen. ” 


Detailed Examples of Districts’ Use of the 5Essentials Survey Data 


Analysis Techniques: 

• SWOT analysis 

• Statement matrix 


• Data coaches with 
teachers on institute 
day 

• SIP team received 
findings and 
interpreted results 
together 

• Started at district, then 
moved to building level 
looking for data trends 


Examples of 5E Data Analysis Process 

Based on the 2013 5E results, a principal conducted a SWOT analysis in September 
2013, examining Strengths, Weaknesses, Opportunities for improvement, and Threats to 
those opportunities. Following the 5E research that indicates strength in three of the five 
essentials should have a positive impact on student outcomes (Bryk et ah, 2010), the district 
selected three Essentials (collaborative teaching, ambitious instruction, and supportive 
environment) as their focal points. The teachers identified approximately 30 areas that they 
felt were weakness or opportunities for improvement, and threats to those opportunities. The 
principal indicated that all of the teachers’ requests to address these areas were fulfilled. The 
school repeated the SWOT analysis for 2014 for the same three Essentials, and the principal 
described the 5E data as “informing our work for the next year. So everything that we do is 
kind of around the 5Essential survey.” 

District administrators shared their 5Essentials results with teachers on the opening day 
of school. At both the district level and building level, they reviewed the data by creating 
a statement matrix that included the 5Essential domain (e.g., ambition instruction), how 
it was measured for various components (e.g., course clarity, quality study discussion) for 
English and math instruction, and the scores. The teachers suggested ways to improve the 
score on each measure. This process also informed decisions about professional development 
throughout the upcoming year. 

A district School Improvement Team member described one of the high schools using their 
last institute day to share the data for their building. They investigated strengths, areas of 
concern, and areas that need improvement, identifying a couple of areas to address. They 
brought in someone to help them understand how to interpret data so that they could “learn 
from the data to try to improve areas in the building.” 


A principal described his school’s process for analyzing the 5E data this way: 


“I lead our School Improvement Team and we actively use that data from the first year’s 
5Essential Survey. This year in our discussion process, I went through the entire data that 
came out. Everyone on our team analyzed it and then we bulleted out what we believed to 
be important significant highs and lows, positives or negatives that seemed to come out of 
the data from both the combination of the student data and teacher data. Our 12 member 


14IERC 2014-2 


Improvement Team is well aware of what we did and how to analyze the information. There 
are some perceptual takes out of any kind of statistical type of data like this. You have to go 
after it with a lens. So we taught each other things about how we perceived [the data] and 
then came to a consensus on what we needed to do with some of those elements. The rest of 
our school saw the roll out with regard to some of the school improvement initiatives. Our 
discussions have been on strengths and weaknesses that seemed to be apparent in the perceptual 
viewpoint of the people who took the survey. ” 

A district administrator described their process for analyzing the 5E data: 

“We started looking at district-level data and see globally how are we doing ? What are our 
strengths and weaknesses? And then we kind of pared down from there. We go to the school 
level, take a look at strengths and weaknesses, and look at the teacher data, the student data, 
and the parent data. And then you kind of gradually draw down and try to find what are 
we good at, what are we not good at, are there consistent patterns across the district, across 
schools? Maybe elementary schools, for example, tend towards being stronger or weaker in some 
areas. Or maybe it’s completely idiosyncratic at the school level. There’s no scripted start at the 
top, work your way down, and kind of see where it leads you. Eventually, I think, what some 
schools — at least last year - they went down to the item level. ” 



Examples of Utilizing 5E Data with Rising Star School Improvement 
Process 

A district utilized the 5Essentials data by integrating it with their Rising Star and teacher 
evaluation data, specifically, in order to set goals for school and teacher improvement. A 
district administrator there noted that the 5E data complements other student data used for 
school improvement planning: 

“We depend more on obviously the quantitative data that’s connected to student outcomes 
because that, in our opinion, gives us more of an idea of where our students are, versus the 
qualitative information the students gave us. So, it’s information. You always look at it and see 
if it matches up. ” 

Similarly, the superintendent from another district worked with all the district’s principals 
to develop goals based on the 5E data. The 5Essentials goals were included with student 
achievement goals and attendance goals. Elowever, due to having only one data point 
throughout the year, he hasn’t found much opportunity to dig into the data on a frequent 
basis. 

The Rising Star Team in a separate district examined improvement areas from the 5E data 
and identified the need to work on teacher collaboration. The team met to brainstorm about 
when teachers feel they have time to collaborate or are able to share information with their 
colleagues. They also used the 5E data to explore ways to improve instruction. 

A superintendent also plans to use the 5E data with the district’s Rising Star process. The 
district staff hope to determine which of the 5E domains align with the Rising Star indicators 
that the schools and district have identified as areas to address. As an example, he described 
examining Rising Star indicators that pertain to school climate and match elements from 
the 5E domain of Supportive Environments. As a next step, he plans to ask, “What do the 
5Essentials tell us about supportive environments that can help inform where we need to go 


Use with Rising Star 

• Compare with other 
student data to see 
where it matches 

• Use 5E to determine 
goals 

• Rising Star team used 
5E data to improve 
instruction 

• Align 5E domains with 
Rising Star indicators 
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Implementing or 
Expanding Programs 


• More opportunities for 
teacher observations 

• Freshman monitoring 
program, freshman 
orientation, and 
resources 

• Parents on school 
committees 


Add parent question to 
the 5E Survey 


from here?” He envisions doing this with each of the 5Essentials domains to glean valuable 
data from survey responses, as well as using the 5E data for “monitoring, and not monitoring 
in a bad way, but monitoring in a growth mindset, to say, ‘Hey, we need to improve on 
this.’” 


Examples of Implemented Program or Expanded Program Based on 5E 
Data 

A principal used the 5Essentials data in school improvement planning “down to every 
question.” Working with the instructional leadership team, they examined questions 
with large differences in response options. They then delved into possible reasons for why 
responses might be at “strongly disagree,” using 85% “strongly disagree” as a cutoff. This 
information helped guide decision making towards school improvement. The principal 
commented, “All of this work is school improvement in any capacity you look at, because 
it all ties into teacher instruction, which then impacts mathematics, which then impacts 
reading, which impacts response to intervention, etc.” The district found that some of the 
discrepancies were accurate and made plans for changes. However, others items were found 
to be inaccurate because the question was misinterpreted by teachers due to how it was 
phrased or due to responses from individuals not frequently in the building (e.g., specialist 
teachers). Through this process of focusing on the “disagree” or “strongly disagree” responses, 
the principal and team identified lack of teacher involvement with other teachers in their 
classroom and communication issues with specialists as areas to address. Based on their 
analysis of the data, more opportunities for teachers to observe other teachers were created, 
and communication with specialist teachers was increased to provide them more support. 

Conversations stemming from teachers after taking the 5E Survey in Spring 2013 prompted 
school administrators and staff to conduct visits to their higher performing schools in the 
district to examine issues concerning freshman success. Based on their experience with the 
5E Survey and information gathered from the school visits, the school administrators and 
staff believed they had strong evidence that more assistance was required for students in their 
first year of high school. Thus, the school implemented a freshman monitoring program 
in Fall 2014. By the end of the first year of the program, the school believed it had been 
“valuable to saving some kids that otherwise might have fallen through the cracks in prior 
years” and is happy to have affirmed the program even though the 5E data were not yet 
available. In addition, a freshman step-up curriculum was implemented this year because 
of their experience with the 5E Survey. The school ensured that their teachers had access 
to information to assist eighth graders with their move to high school next year. Working 
with the guidance office, information about what happens in high school was provided to 
students. The principal also added, 

“We have plans to explode a freshman orientation that we do in August into a much bigger 
event because of that data as well. So that’s probably a good example of something that we put 
in place as a result of the 5Essentials survey. ” 


Based on 5E results from parent surveys, administrators and teachers decided to modify and 
increase outreach activities in order to encourage parents to become more involved with 
the schools. Now, parents in the district are invited or nominated to be part of more school 
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committees. In addition, training is offered so parents “can be involved on committees in a 
meaningful way so that they can voice their opinions and feel involved.” 

A district administrator at a different district also commented their district is “completely 
invested in doing this [the 5E Survey] In fact, the district added an additional question 
to the survey asking parents to rate/grade their schools. This question was added to keep 
continuity with other surveys the district conducted prior to the 5Essentials. This district 
administrator felt the local School Board had a strong interest in soliciting community input 
and noted, “It seems like a natural way to involve parents.” 

Reasons for Not Using the 5Essentials Survey Data 

For respondents that stated they did not use the 5Essentials survey results or utilized the 
data despite concerns, we also asked why they chose not to use the data results from the 
5Essentials Survey. The responses fell into four main categories: credibility of data, districts 
already utilized a climate survey, overwhelmed by other mandated activities, and other 
miscellaneous issues. Nearly everyone interviewed expressed some concern about the 5E data. 
Many of these districts still utilized the data, working within the constraints they perceived, 
while other districts simply did not utilize the data due to their strong concerns. 

Credibility of data. Many participants, from all of the districts and regardless of participant 
type, expressed some level of concern about the credibility of the 5E Survey data. Lack of 
credibility was described in many different ways, but all resulted in some participants’ lack of 
trust in the accuracy of the data. 

The most cited reason related to credibility of the 5E data was the potential for multiple 
entries by the same person or completing the survey for schools other than their own. 
Somewhat related was the issue of survey results being deemed inaccurate when the reported 
teacher response rate did not match the actual experience of the school or a response rate was 
over 100%. 

“ But that’s a concern that I could have gone on as a student and signed on — anyone could. 
Anyone could sign on as a parent. So those were things that were troublesome because you’re 
not getting good data. ” 

“Data security really hindered the process. Because then we didn’t trust it. ” 

“Teachers are also concerned that since students and families can take the 5Essentials more 
than once theoretically, that the most disgruntled people could skew the results by multiple 
instances of participation. ” 

Although the University of Chicago Impact (UChicago Impact) gave assurances that the 
duplicates and erroneous entries were deleted, many individuals were still skeptical of the 
data due to other survey administration and reporting issues. 

Problems with survey language were the second most cited reason for diminished 5E data 
credibility in the eyes of the participants. Examples of problematic survey language included: 
confusing wording such as “target class” and “self-contained teacher” (especially for students 
and parents), the use of “always,” the lack of a “does not apply” option, and inclusion of 
questions that did not pertain to their district/school. Several individuals questioned whether 


Concerns about 
credibility of the 5E 
data were cited most 
often for not utilizing 
the 5E results. 
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survey items outside the control of the district (e.g., safe communities) were valid to include 
in ratings about the district and schools. 

“I really think the way some of the questions are worded, I think it can really skew the data. ” 

“Some of the other issues are when we talk about the measurement for involved families. 

There’s some assessment about the community being safe and the neighborhood park being safe. 
Those are things that are outside the control of the school district, yet they’re reflected as if it’s 
our organization. That’s a concern for me. I’m being rated and then our organization is being 
rated and that rating is being publicized and it’s not something that I control. ” 


Concern about generalizability of the survey also decreased credibility. Due to low 
response rates, particularly from parents, many individuals felt the survey results did not 
accurately reflect the opinions of the entire group surveyed. Also, individuals’ understanding 
that the 5E Survey was developed from the Chicago Public Schools elicited responses such 
as “How applicable is this to us?” signifying individuals’ skepticism that a single survey could 
address the individual needs of all districts/schools throughout the state. These individuals 
often commented that “one size doesn’t fit all,” and that schools were too different to 
compare results, with intense disagreement with the decision to norm the survey results to 
Chicago Public Schools’ scores. 

“A survey that was developed to analyze climate in Chicago Public Schools is certainly not 
going to fit rural, central Illinois. It’s just not going to make it. ” 

“That’s why we had some low parent turn out. They just didn’t know what some of those 
questions were and how to even begin to answer them. ” 


The sentiment “One 
size doesn’t fit all” was 
reported frequently 
and reduced use 
of the 5E data. 


Smaller districts and districts in the central and southern regions of the state cited these 
concerns most often; however, a few larger districts in the northern part of the state also 
expressed opposition to being compared to Chicago Public Schools. 

Other implementation issues also had a negative impact on the credibility of the 5E results. 
Technology problems with logins, sessions timing out, and inadequate number of computers 
to efficiently administer the survey hindered some districts. Participants at several districts 
indicated that unclear survey instructions for parents with multiple children or students with 
multiple teachers may have decreased response rates. Also, lack of definitions concerning to 
whom survey items referred (e.g., principal versus assistant principal, teacher versus specialist 
teacher) caused some survey respondents confusion, thus resulting in potentially inaccurate 
responses and lack of confidence in the results. 


A few administrators commented that they needed more data over time before they would 
consider utilizing the 5E data for school improvement or other decisions. Administrators and 
teachers of K-6 buildings expressed concern about their buildings’ results being based only 
on input from 6th grade students, and thus, producing a misleading representation of their 
entire building. A few participants advocated for additional surveys for younger grades to 
more accurately describe their building. 


District already utilized a climate survey. Most (11 out of 15) districts we interviewed 
indicated they already administered a locally-developed or national (e.g., National School 
Climate Center) survey to their constituents that addressed school climate, which they felt 
met their needs. A small number of administrators were frustrated that the State Board of 


18 IERC 2014-2 


Education did not appear to consider the option of districts using their local surveys as a 
valid alternative to the 5E Survey (note that Public Act 098-0648 initially dictated ISBE to 
administer one statewide survey). Due to the State mandate to administer the 5E Survey 
and the districts’ desire to continue their own survey, some administrators and SIP members 
believed survey burnout may have occurred, possibly resulting in lower response rates or 
erroneous responses for the 5E Survey. Some district leaders decided not to administer their 
local survey to reduce survey burden, and thus felt that they lost valuable data. 

“We have done internal surveys for years. I think part of the reason why the 5Essentials hasn’t 
been promoted more is because we use our own internal information and we have used it 
for so many years that when we were presented the idea of the 5Essentials, we had already 
completed parent surveys, we had already completed teacher satisfaction surveys, and then we 
were asked to do the 5 Essentials. ” 

Overwhelmed by other mandated activities. Participants from most of the districts 
(particularly small and mid-sized districts) expressed exasperation at the large number of 
other activities (e.g., new administrator and teacher evaluations, PARCC, Common Core) 
they were mandated to implement, and, in these instances, the 5E Survey was often viewed 
as just one more burden, particularly during the spring timeframe. Interviewees reported 
that many teachers may not have completed the survey due to feeling overwhelmed by other 
responsibilities. In addition, the 2014 administration window for the 5E Survey coincided 
with teacher evaluations and reduction-in-force notices in several of the districts we visited. 
We heard from many individuals in these districts that they believed this may have influenced 
how teachers answered survey questions, particularly those relating to their principals. 
Administering the 5E Survey close to the ISAT testing timeframe also created problems. 
Several individuals were unhappy that the time students needed to take the survey reduced 
instruction time. All of these related concerns created an atmosphere in which completing 
the 5E Survey and utilizing results was seen as very low priority. 

“Well, basically, the teachers are overwhelmed. The kids are overwhelmed. We’re trying to 
squeeze everything in. ” 

“The timing of the school year with the survey is very difficult because around this time, and 
especially with the high school, you’re really looking more into the seniors completing things 
as well as state testing time and the PSAE window. The 5Essentials survey, even getting 
the results back is not something that is even in the timely manner we probably got the 
information back late April or May. That’s not like your hierarchy of needs at that time of the 
school year. ” 

Other Miscellaneous Issues 

Not encouraged to use 5E data. Some participants from about a third of the districts 
indicated they did not use the 5E data because they were directed or encouraged not to do so 
by school administrators. Some administrators reported they did not encourage use of the 5E 
survey data or did not disseminate the data because of their own concerns about the validity 
of the 5E data and because other local climate survey data were available that they believed 
were more accurate. 

“Since the determination was made that the scores did not appear to be valid for us, it wasn’t 
shared. ” 






Perceptions that 
5E Survey did not 
meet local interests 
and needs hindered 
districts’ use of 
the results. 




“I think the timing of it is not 
a good time. They are doing 
this at time that is, every 
year, never fails, we are doing 
it in March. You are doing 
it at a time when we are 
doing ISATs. You are doing 
a time when that is when 
RIF notices go out. You are 
doing it at a time where this 
year and the last two years we 
have had to do major cuts. ’’ 
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Other concerns about 
confidentiality in small 
districts, uncertainty 
of 5E Survey, 
misuse of result, 
funding priorities, 
and publicizing 
the data were 
expressed less often, 
but none the less, 
negatively impacted 
use of 5E data. 


Small population size issues. A few of the smaller districts raised concerns about utilizing 
the 5E data when the district size is so small that confidentiality of staff is difficult to 
maintain or when the threshold to report the 5E data can be hard to reach due to the small 
number of staff employed at the district. In addition, some individuals in these districts 
worried that responses from a small number of disgruntled teachers or students could carry 
more weight in their results due to the smaller size of the overall population surveyed. 

“It is harder in a small school to keep the confidentiality. ” 

“It doesn’t make any sense to require a district to do it and then they can’t even get their results 
back because they’re a small district. Don’t penalize the small districts. ” 




“My other concern is that our small population — you know, one student is a pretty high 
percentage in terms of response, as is one teacher. 


Lack of information about the 5E Survey. We heard from a few participants at several 
districts that they do not know enough about the 5E Survey to use the data or to encourage 
others to use it. Their comments and questions ranged from “What is the purpose/goal?” to 
“Explain the data reliability, validity, and analysis?” to “What to do with the data?” Another 
participant wanted more explanation about how the survey would benefit their school and 
community. 


“I felt it was a gotcha, to 
be quite honest with you. I 
thotight they were looking 
to make us, the State Board 
or whomever, that was the 
driving force behind this 
thing was trying to make us 
look bad as public schoob. ” 


“I think just the main thing is we don’t see the purpose in it. We don’t see how it would benefit 
our school. We don’t see how it would benefit our community because it’s not necessarily 
completely aligned to what we work for, nor our goals. So because of that, we don’t invest a lot 
of effort into looking at it and looking at the data and the change in data possibly. ” 

Concerns about use for evaluative purposes. We also heard concerns from several 
superintendents that their principals and others around the state felt that the 5E Survey 
would be used as an evaluation tool for principal performance. Principals were concerned the 
5E data would be misinterpreted in their evaluations, particularly in cases when teachers may 
have responded negatively due to low evaluation ratings, reductions-in-force, or substantial 
reforms for school improvement purposes. 

Concerns about funding. A couple of superintendents conveyed strong objections about 
the State using funds on the 5E Survey when budgets were being cut in many areas and, in 
their view, insufficient support was provided for implementing the 5E Survey. They expressed 
frustration, particularly because the 5E Survey had implementation problems, that another 
unfunded mandate was required of districts. 


Making the 5E results available to the public. A few individuals expressed concern about 
making the 5E data results available to the public, especially in light of the issues addressed 
above. In addition, there were concerns about whether parents and community members 
would be able to accurately interpret results and not knowing the full context of the district 
or school situations that may have influenced the responses of students, teachers, and parents. 
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Suggestions for Improving 5Essentials Survey 

Respondents made numerous suggestions for improving the 5Essentials Survey, many of 
which were related to the aforementioned concerns that limited 5E data use. In this section 
of the report, we summarize and organize these suggestions into a coherent framework, 
beginning with recommendations for improving the survey and its administration, and then 
moving on to recommendations for improving the results of the survey and their utilization 
for school improvement. We provide a few examples of comments from participants to 
illustrate each point. We highlight the most common recommendations for improvement in 
this section, and provide additional respondents’ suggestions in Appendix B. It is important 
to note here that we do not necessarily endorse all of these suggestions, but rather strive to 
report the range of (sometimes contradictory) recommendations from the field. We leave it 
to the reader to weigh the pros and cons of each. 



Improve the 5Essentials Survey Administration 

1. Market the survey — frame and motivate participation and utilization — in order to 
increase buy-in. 

Respondents suggested that schools and districts needed to be provided with a reason to 
use the 5Essentials Survey and training to help them understand its empirical backing and 
potential benefits, particularly its relationship to school improvement. In the words of one 
respondent: “Provide us something that meets a need and we ’re probably all ears.” Another 
respondent commented: 

“I haven’t had a full day of training on 5Essentials yet, and much less the kind of training that 
we got for the capacity builders training, where they’re training you how to train other people 
on it. We need that and we need that done really effectively at the state level, because again, 

I see a lot of value in this data. I know that it won’t get used well unless the districts have the 
ability to handle it themselves and understand it enough to support it, so we wouldn’t be using 
Rising Star in our district as much if we weren’t trained as capacity coaches. ” 

Respondents also suggested that such training ought to occur prior to rolling out 
implementation, and recommended working with professional organizations (such as the 
Illinois Association of School Boards (IASB) and the teachers unions) to attain their support 
and assistance. Respondents stressed that it was important to be clear about the purposes of 
the survey and re-emphasize that it is intended to be helpful, and not punitive. 

Additional “marketing” ideas included creating tools such as video clip, talking points, and 
power point slides to share with educators and parents — describing the survey and how it 
can be used. Respondents also suggested keeping in mind that families also have to be sold 
on the survey — they need to understand why they’re being asked these questions, and schools 
should let them know that “this is just one way that we value your input.” Some respondents 
suggested that ISBE should “have a stronger message” and convince districts that the 5E is 
superior to alternative climate measures. Elowever, other respondents recommended that 
ISBE should allow districts to use (or continue to use) alternative climate measures that 
are also research-based and agreed upon through collective bargaining, granting districts 
flexibility similar to teacher evaluation regulations under the Performance Evaluation Reform 
Act (PERA). 


“Provide us something 
that meets a need and 
we’re probably all ears. ’ 


“We significantly increased 
our response rate by having 
parents fill out paper surveys 
that were scantronned. ” 
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. . if it’s just an every other 
year you take the survey 
. . . it’s almost like it’s not 
important in that manner. ’’ 


if they would say, ‘add this 
to your school improvement 
plan, take this data, here are 
the places you’re going to put 
it in’, then it will get done. ” 


“. . . that the model seems to 
be very elementary-driven, 
which turns off the high 
schools right away. So maybe 
if it’s a high school version 
and an elementary version. ” 


“I think there should be 
a different type of survey 
for parents that would be 
more conducive for them 
because they don’t know 
what the curriculum is. 
They don’t know what 
their kids are learning on 
a daily basis. Other than 
meeting the teachers at 
parent-teacher conferences, 
or maybe through emails, 
they probably don’t know 
a lot about the teachers. ” 


2. Increase breadth of participation and response rates 

Numerous strategies were reported for increasing participation, particularly for parents, 
including handing out surveys when parents pick up their children from school or offering 
pizza parties when students bring in completed parent surveys. Similarly, respondents 
suggested creating a modality other than online to complete the survey, given that not 
all parents have computer access. Other respondents recommended making the survey 
less challenging to complete online, especially on a mobile phone. Some respondents 
viewed the response rate threshold for parents as unrealistic and suggested ISBE reduce 
the proportion of parents that must participate in order to view results (though several of 
these respondents also recognized that certain response rates might be necessary to ensure 
validity). Several respondents suggested that emphasizing the importance of the survey 
and informing participants that results will be posted on the school report cards could 
encourage respondents to consider their responses more seriously. Interviewees also noted 
that it was important for all staff members to be included in the survey process, including 
administrators. Several recommended that administrators should “at least take the survey 
as an idea of what the teachers, parents, and students are doing, or some variation of it.” In 
addition, several respondents suggested that requiring the survey annually and integrating it 
more tightly with the school improvement planning process could increase participation and 
utilization. 

3. Rostering 

To address the issue of potentially invalid entries, many participants suggested using a 
rostered survey to ensure that participants do not take the survey more than once or for the 
wrong school. They noted that rostering would also provide survey administrators with the 
ability to determine who still needs to complete the survey so they could target reminders 
or check the representativeness of local responses. Others cautioned against using a rostered 
survey due to concerns about the perceived loss of anonymity. 

“I think the rostering thing, although there’s benefits, I wonder if that’s going to end up being 
seen as less anonymous, because suddenly, you have your own unique code and I can tell if you 
used it. Or when you used it or where you used it from. There’s a lot of crazy information that 
can be gleaned from behind that. So I wonder if that — there are pros and cons with that to 
me. ” 


4. Improving the survey items 

Many suggestions regarding specific items on the survey were provided. These included: 
eliminating questions that pertain to the community; creating different versions of the 
survey for elementary and high schools; creating a more parent-friendly version; creating a 
version completely in Spanish; and modifying terminology such as “picking up the report 
card” and “self-contained.” Respondents also provided advice about the design of survey 
response categories, such as the addition of a “does not apply” option, a “don’t know” option 
with the opportunity to include comments, and further guidance about the differences 
between response categories. Some respondents recommended that ISBE continuously revisit 
survey items and seek feedback from schools about misinterpretations, confusion, or any 
technological issues that arise. 
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Other suggestions for making the survey more user-friendly included adding a progress bar 
to the online survey, providing login assistance, making sure respondents have correct login 
information, or simplifying usernames and passwords. Many respondents suggested making 
the survey shorter to help ensure respondents are taking the survey seriously and to increase 
response rates. 

5. Timing of the survey 

Numerous respondents felt the current survey administration window should be moved 
earlier in the school year — but not so early that respondents cannot accurately respond to 
items. These interviewees suggested that earlier survey administration could ensure that 
results would arrive sooner and be more actionable. Also, several participants suggested 
that ISBE avoid overlap between 5E’s administration and other mandated state assessments 
(ISAT, PSAE, PARCC) to the extent possible. Some respondents noted that allowing a longer 
administration window and more flexibility as to when the survey is administered may also 
improve response rates, particularly for parents. 

“If we could have the parent timing at different times, to be able to do it on our own schedule 
— when we need to do it. I think that would be better. Like I said, when I get my parents in 
the building. . . ” 

Improving the Presentation Usefulness 

1. Timing and Format of Results 

Many respondents wanted more immediate feedback from the 5Essentials, preferring that 
results arrived prior to the beginning of the next school year to allow schools to address issues 
over summer break. Several respondents recommended disseminating the results more widely 
(many were not aware the results were available on the school report card) and making it 
easier to access the results. Some respondents also noted that they would prefer the data to 
be presented as a series of charts and graphs, rather than a long report or binder full of data, 
so that it would be more easily understood by parents, community members, and the school 
board. Elowever, some more data-sawy respondents requested increased access to within- 
school details in the data, or even raw data that would allow them to disaggregate responses 
within schools. For example, some administrators were interested in seeing the differences 
between veteran teachers and novices in their buildings, or between different grade levels or 
subject areas. 

2. Putting the data in context (reporting normed scores vs. frequency distributions) 

In the absence of norm-referenced results, some respondents found it difficult to interpret 
results and place them into context. Thus, many respondents encouraged the use of normed 
and color-coded results to help make sense of the data and determine whether and where 
their results were relatively strong or weak. 

Others suggested that the results not be color-coded, because of the negative reactions this 
could provoke from teachers and the public, especially in an era of high-stakes accountability. 

“The initial presentation of the data caused so much emotion that our staffs response, major 
response was to shut it down. It’s not useful at all. ” 


“It’s just like the Prairie 
State Exam, or any type of 
assessment that’s happening. 
If we don’t get the data 
back in a timely fashion 
where it could be used for 
real improvement, it’s not 
going to have that value. ” 


“We took the raw data 
files and created Excel files 
and plugged in each of the 
SEssentials and the individual 
questions, then broke that 
down by school so that we, 
as a central office staff, could 
compare schools and then 
we could compare that to 
the district’s average. So, 
what could improve ? How 
the data is recorded, because 
we had to do some work 
to make it presentable. ” 


“If we had things to compare 
it to about how other students 
in other schools in the state are 
doing that would be great. 


w 


“One thing that I would be 
looking for, though, woidd 
be districts that would have 
comparable demographics, 
comparable issues, so that 
we could benchmark against 
other districts and potentially 
do some sharing of why 
people get the results they do 
and what to do about it. ” 
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“I think that kind of 
data is really to improve 
yourself, and it's almost 
sometimes like airing your 
dirty laundry out there. ” 


Some respondents also suggested that their scores could be benchmarked against 
demographically similar districts or their own scores over time, rather than statewide (or CPS- 
based) norms. 

Numerous respondents also requested assistance in determining how their responses were 
converted into normed scores, and were particularly puzzled as to how high proportions of 
“positive” responses could convert to a “negative” result after norming. 


3. Public vs. private reporting 


In a similar vein, several respondents suggested that the 5Essentials data should be for school 
improvement use only, and not reported publicly. They argued that making the 5Essentials 
data public (through state report cards, the news media, Facebook, etc.) — particularly color- 
coded normed scores — could shift the focus from productive to punitive. 


“I don’t think a lot of people 
have had training on just how 
to read data, how to interpret 
what we’re getting back in the 
classrooms and how to make 
it applicable in our plan. ” 


“I think that kind of data is really to improve yourself, and it’s almost sometimes like airing 
your dirty laundry out there and you could deter people from ever wanting to come to our 
school if they saw a survey where teachers are very upset, not understanding it’s because we’re 
making such changes in the stride to education. So I feel like, you’re hurting schools that are 
trying to make that right change instead of helping move them forward. Because those looking 
at the 5E results aren’t seeing, ‘OK this is where we started and this is where we’re going. ’” 

Others suggested that administrators should be provided with space to respond to the results 
or provide narrative context, as is done on some other ISBE reports. 

4. Provide more actionable feedback 

In order to increase utilization of the 5E results, many respondents expressed the need for 
training and explicit directions for using the data. Some respondents would prefer a more 
formal link between the 5Essential Survey and school improvement plans. For example, some 
respondents recommended that ISBE could match districts with similar weaknesses and 
recommend proven strategies from districts that had experienced improvement or success. 
Respondents also suggested that more training in data analysis and utilization was required to 
help them understand and use the survey results. 


Districts are looking for 
suggestions for “next 
steps” to help them 
utilize the 5E data. 


Similarly, several respondents also noted that time is needed to be set aside to review and 
interpret the survey results as a school- or district-wide team. Other respondents requested 
assistance in using the 5E Survey results to easily identify problems or emerging issues at their 
schools. 

“That’s something Rising Star does after we assess it. It gives everything a number. And it tells 
us what we think it is, without knowing it, we assessed everything and it gives it a number. 
And so we can then see what’s more important. So that might be something just so that it’s not 
just like, here’s all of this. It kind of helped determine what would be the best place to spend our 
efforts first. ” 
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Lastly, respondents would like more concrete advice for “next steps” about how to improve 
weaknesses identified by the 5Essentials Survey results. They suggested that ISBE (or some 
other entity) could work to develop training, tools, road maps, or examples that schools and 
districts could use to improve their results. 

“But were at the point now where we need help with the implementation, having to help 
principals. How do I use this information to put into my plans l What does going from yellow 
to red and green look like in terms of a lesson plan ? You know, boiling it down to that level. ” 

“I guess I would like ideas from the State — one-page idea of what to look at, how to look at 
it, and how to implement it. ” 

“I guess, if there’s other schools or school districts out there that are currently using it to drive 
professional development choices, school improvement, I think knowing what other schools are 
doing with it and how other schools are using it would be good. ” 

Analysis of ISBE Implementaion Feedback Survey Data 

In January-February 2014, ISBE surveyed superintendents and principals from all Illinois 
public schools about their opinions of and experiences with several ISBE initiatives, 
including the 5E Survey and data reports. The number of completed surveys was 273 for 
superintendents and 634 for principals, with estimated response rates of 32% and 16%, 
respectively.' Respondents were asked to indicate their level of agreement on a four-point 
scale (Strongly Agree to Strongly Disagree) to questions concerning the 5E teacher survey, 

5E student survey, 5E survey reports, and use of the 5E data. We analyzed these data with 
descriptive and inferential statistics using percentages, cross-tabulations, and chi-square 
analyses. Familywise error rate was corrected using Benjamini and Elochberg’s (1995) false 
discovery rate to balance concerns of inflated Type I error when multiple tests are conducted 
and to allow maximum exploration of the data. A separate correction was conducted for 
each subscale of the survey items. Favorable and unfavorable opinions were determined 
by aggregating strongly disagree and disagree responses and aggregating strongly agree and 
agree responses. The response patterns of principals and superintendents were compared 
and differences by region and district size are noted, where significant. Survey respondents 
indicated the region in which their district/school was located: Cook County, suburban 
Chicago, northern Illinois, central Illinois, and southern Illinois. Respondents also indicted 
the enrollment of their district: <300, 300-599, 600-999, 1,000-1,499, 1,500-2,000, >2,000. 
We collapsed these into three categories for analyses purposes: small (<600), medium (600- 
2,000), and large (>2,000). See Appendix C. 


Superintendents 
and principals were 
favorable on: 

• survey communication 

• teachers’ willingness 
to complete survey 

• few logistical 
challenges. 


For the teacher survey, 
superintendents 
and principals were 
unfavorable on: 


• value of teacher 
survey data 

• questions addressed 
local interests. 






Majority of principals 
felt the teacher 
survey administration 
produced fair, 
reliable data, while 
the superintendents 
did not. 




w 


~ Response rates were estimated based on data from the ISBE Directory of Education Entities for 2013-14 which 
reported 865 districts and 4008 schools. 
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5Essentials Teacher Survey Feedback 

Regarding the series of questions focusing on the teacher survey component of the 
5Essentials, both principals and superintendents provided favorable ratings regarding 
whether communication was sufficient to administer the survey (70% and 59%, respectively) 
and whether teachers were willing to participate in the survey (79% and 67%, respectively). 


Teacher survey generated data I found valuable. 


Communication was sufficient to administer survey. 


Superintendent 

Principal 


Teachers were willing to participate in survey. Superintendent 


to generate fair, reliable data. 


issues of interest to my district/school. 


challenges due to technological issues. 


survey administration for other reasons. 



Mean 

2.13 
3.7% 2.37 

2.51 
2.67 

2.67 
2.88 

2 . 2 % 2.11 
2.43 

2.22 


2.18 


■ Strongly Disagree (1) ■Disagree (2) Agree (3) b S trongly Agree (4) 
Figure 1. Views on the 5Essentials Teacher Survey: Differences by Leadership Role 


The responses 
provided by the school 
principals indicated 
they held more 
favorable, or less 
unfavorable, views 
of the teacher survey 
relative to district 
superintendents 
across all items. 


Neither principals nor superintendents believed that the district/school encountered 
significant logistical challenges due to technical reasons (13% and 18%, respectively). 
Likewise, neither principals nor superintendents believed that the district/school encountered 
significant challenges due to other reasons (19% and 31%, respectively). Both principals 
and superintendents provided unfavorable ratings regarding whether the teacher survey data 
was valuable (51% and 64%, respectively) and also provided unfavorable ratings regarding 
whether most of the survey questions adequately address issues of interest to their district/ 
school (52% and 59%, respectively). Superintendents rated whether survey administration 
procedures were adequate to generate fair, reliable data unfavorably (64% unfavorable) while 
principals rated this item favorably (53% favorable) (see Figure 1). 

Leadership Role Comparisons 

The responses provided by the school principals indicated they held more favorable, or 
less unfavorable, views of the teacher survey relative to district superintendents across all 
items (see Figure 1). These differences were statistically significant for all items except for 
two items: whether the survey adequately addresses items of local interest at the school 
and whether the district encountered significant logistical challenges due to technology. 

For the former item, both principals and superintendents had unfavorable ratings and 
for the latter item, both principals and superintendents had favorable ratings. The largest 
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percent difference in opinion between superintendents and principals was on whether 
survey administration procedures were adequate to generate fair, reliable data, in which the 
principals were more favorable by 17.5 percentage points. 

Regional Comparisons 

There were no statistically significant differences in favorability ratings of the teacher survey 
items based on region for either superintendents or for principals. However, the descriptive 
findings from the sample of superintendents and principals in this study were: 

• As a group, the responses from the southern region of Illinois were somewhat unique 
in that the superintendents generally had more favorable views of the teacher survey 
and fewer reported logistic challenges when compared with principals from the same 
region. Among all other regions, the pattern was the opposite in which the views of 
principals regarding the teacher survey were generally more positive. 

• For nearly all of the teacher survey items, disagreement was greatest between 
principals and superintendents from Cook County. 


No statistical 
significance 
difference between 
superintendents 
and principals 
based on region. 


Principals from large 
districts had more 
favorable views on 
the value and local 
interest of the teacher 
survey items. 


Enrollment Size Comparisons: small (<600), medium (600-2,000), and 
large (>2,000) 

The superintendent’s ratings of the teacher survey items were not statistically significantly 
different depending on what size district they oversaw. However, there were two statistically 
reliable differences among principals across different sized districts: 

• There were significant differences in attitudes regarding the value of the teacher 
survey items depending on district size. Large districts had the highest percentage of 
principals indicating that the teacher survey data was valuable (57%) while medium- 
sized districts had the fewest principals indicating that the teacher survey data were 
valuable (40%). 

• There were significant differences in attitudes regarding how the survey items 
reflected local interest. Large and small districts had the highest percentage of 
principals indicating that the teacher survey items reflected local interest (55% and 
53%, respectively) while medium-sized districts had the fewest principals indicating 
that the teacher survey items reflected local interest (35%). 


For the student survey, 
superintendents 
and principals were 
favorable on: 

• survey communication 

• students’ willingness to 
complete survey 

• few logistical survey 
challenges. 


5Essentials Student Survey Feedback 

In terms of the questions specific to the student survey component of the 5Essentials, both 
principals and superintendents provided favorable ratings (73% and 66%, respectively) 
regarding whether communication was sufficient to administer the survey and whether 
students were willing to participate in the survey (73% and 69%, respectively). Only about 
one-fifth of the principals and superintendents believed that the district/school encountered 
significant logistical challenges due to technical reasons (18% and 22%, respectively). 
Likewise, approximately one-fourth of the principals and superintendents believed that 
the district/school encountered significant challenges due to other reasons (22% and 25%, 
respectively). Both principals and superintendents provided unfavorable ratings regarding 


Superintendents 
and principals were 
unfavorable on: 


• value of student 
survey data 

• questions addressed 
local interests 
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whether the student survey data was valuable (56% and 51%, respectively) and also provided 
unfavorable ratings regarding whether most of the survey questions adequately address issues 
of interest to their district/school (56% and 58%, respectively). Superintendents did not 
view that survey administration procedures were adequate to generate fair, reliable data (56% 
unfavorable) while principals rated this item favorably (54% favorable) (see Figure 2). 


Student survey generated data I found valuable. 


Communication was sufficient to administer survey. 


Students were willing to participate in survey. 


Survey administration procedures were adequate 
to generate fair, reliable data. 


Most of the survey questions adequately address 
issues of interest to my district/school. 


District/school encountered significant logistical 
challenges due to technological issues. 


District encountered significant challenges with 
survey administration for other reasons. 


Superintendent (n=261 ) 
Principal (n=434) 


19 . 5 % 


15 . 9 % 


36.4% 


42.5% 


34.8% 


45.6% 


Superintendent (n=261 ) 
Principal (n=431) 


Superintendent (n=259) 
Principal (n=429) 



22 . 2 % 


20 . 2 % 


24.3% 


20.3% 


62.5% 

66.4% 

64.1% 

66 . 2 % 


Superintendent (n=260) 
Principal (n=430) 

Superintendent (n=259) 
Principal (n=429) 


20 . 0 % 


15 . 3 % 


35.8% 


41.9% 

i 


30.5% 


49.8% 


Superintendent (n=261) 
Principal (n=428) 


Superintendent (n=261) 
Principal (n=427) 




Mean 

1.5% 

2.26 

3.7% 

2.37 

3.4% 

2.57 

5.8% 

2.70 

4.6% 

2.66 

6.5% 

2.72 

2.3% 

2.27 

4.4% 

2.43 

0.4% 

2.25 

2.8% 

2.30 

3.1% 

2.01 

4.7% 

1.96 

5.4% 

2.10 

4.7% 

2.05 


100% 


■ Strongly Disagree (1 ) ■ Disagree (2) Agree (3) ■ Strongly Agree (4) 


Figure 2. Views on the 5Essentials Student Survey: Difference by Leadership Role 


Majority of principals 
felt the student 
survey administration 
produced fair, 
reliable data, while 
the superintendents 
did not. 


Leadership Role Comparisons 

Although again, principals seemed to have more favorable views when compared with 
superintendents (see Figure 2), these differences were not large enough to be statistically 
reliable. In this sample of superintendents and principals the largest difference between 
them was on their agreement of whether the student survey items were fair and reliable, 
in which 54% principals agreed and 44 % superintendents agreed. Generally speaking, 
there was greater agreement between the superintendents and principals in terms of their 
responses related to the student survey, than with the teacher survey. This was mostly due to 
the superintendents providing slightly more positive responses specific to the student survey 
administration, relative to how they responded to the teacher survey; whereas principals 
maintained a fairly consistent response pattern across two portions of the survey. 


Regional Comparisons 

Superintendents’ opinions on whether communication about the student survey was 
sufficient were significantly different by region. Superintendents in the Chicago suburbs had 
the most favorable response, with 89.2% agreeing with this statement, while superintendents 
from central Illinois had the least favorable response with only 53.2% in agreement. Across, 
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all other items, there were not statistically significant differences based on region. Principals’ 
opinions on the student survey were not statistically significantly different based on their 
region of the state. 

Enrollment Size Comparisons: small (<600), medium (600-2,000), and 
large (>2,000) 

Superintendents’ opinions on the student survey did not statistically significantly differ across 
district sizes. However, principal opinions about whether the student survey was fair and 
reliable and whether the items reflected local interest were statistically significantly different 
depending on their district size. Principals from large districts had the most favorable ratings 
on each of these items (63% and 54%, respectively) while principals from medium-sized 
districts had the least favorable ratings. Descriptive findings from this sample of principals also 
indicated that: 

• Principals from small schools had the highest proportions in agreement with the items 
specific to the communication (81%) and the willingness of their students to take the 
surveys (83%). A relatively high proportion also found the student survey fair and 
reliable (59%) and related to local issues (45%). 

• Principals from medium-sized schools generally had the least favorable ratings of the 
student survey, especially local interest (35%). 


Principals from large 
districts were most 
favorable about fair 
and reliable student 
survey data and 
items reflecting 
local interest, while 
medium-sized districts 
were least favorable. 


5Essentials Report Feedback 

Principals and superintendents alike generally provided less favorable responses to questions 
about the 5E Survey reports relative to the parts pertaining to their views of the teacher and 
student surveys. On every item, most of the respondents either disagreed or strongly disagreed, 
indicating unfavorable views of the 5E survey reports (see Figure 3). The lowest rated items 
were whether the teachers and parents in the district found the information valuable; 82% 
of the superintendents and 72% of the principals reported that their teachers did not find 
the survey reports valuable. Likewise, 89% of the superintendents and 77% of the principals 
reported that their parents did not find the survey reports useful. 


Survey reports provided valuable information 
in my role as superintendent/principal. 


Survey reports provided new information in my 
role as superintendent/principal. 


Superintendent (n=262) 
Principal (n=562) 


32.4% 


19.0% 


42.0% 


38.1% 


Superintendent (n=262) 
Principal (n=564) 


29.4% 


17.9% 


43.9% 


42.4% 


Survey reports provided easy to use information 
in my role as superintendent/principal. 


Superintendent (n=261) 
Principal (n=564) 


26.4% 


16.1% 


42.5% 


42.0% 


Teachers in my district/school found the survey 
reports valuable. 


Superintendent (n=261) 
Principal (n=550) 


33.3% 


22.5% 


49.0% 


48.9% 


Parents in my district/school found the survey 
reports valuable. 


Principals and 
superintendents 
reported unfavorable 
views about the 
5E reports. 


24.4% 


38.4% 


26.3% 

37.1% 


30.3% 

38.1% 


17.6% 


27.5% 


Superintendent (n=257) 
Principal (n=522) 


36.6% 


24.1% 


52.1% 


10.90 


52.5% 


21 . 8 % 



Mean 

1.1% 

1.94 

4.4% 

2.28 

0.4% 

1.98 

2.7% 

2.24 

0.8% 

2.05 

3.7% 

2.29 

0.0% 

1.84 

1.1% 

2.07 

0.4% 

1.75 

1 .5% 

2.01 


0% 


20 % 


40% 


60% 


80% 


100 % 


■ Strongly Disagree (1 ) ■ Disagree (2) 

Figure 3. Views on the 5Essentials Survey Reports: Difference by Leadership Role 


Agree (3) ■ Strongly Agree (4) 
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Principals in Cook 
County indicated 
more positive views 
on whether their 
parents found the 
5E reports useful. 


Although, both superintendents and principals provided unfavorable ratings, principals 
had statistically significantly more favorable ratings than superintendents on every item. 

For instance, while 74% of superintendents disagreed that the reports provided valuable 
information to them, 57% of the principals disagreed that reports provided valuable 
information to them, a 17 percentage point difference. 

Regional Comparisons 

There were not any statistically significant differences in superintendents’ views of the 5E 
reports by region of the state. However, in this sample of superintendents, those from Cook 
County and southern Illinois generally had the most positive views regarding the use of the 
5E reports. Principals had statistically significant different views on whether their parents 
found the survey results useful based on region, with those from Cook County indicating 
more positive views (38% in agreement) on this item than the other regions (16% to 21% 
agreement). On the other items, pertaining to the value and ease of use of the reports in 
the school, there were not any significant differences by region of the state. However, in 
this sample of principals, those from Cook County consistently provided more favorable 
opinions on the 5E reports than principals from other regions. 


Principals from larger 
districts reported 
more favorable 
ratings on the value 
of the 5E reports. 


Enrollment Size Comparisons: small (<600), medium (600-2,000), and 
large (>2,000) 

There were not any statistically significant differences in opinions toward the 5E reports 
among superintendents of different size districts in the state. Yet, there were statistically 
significant differences in opinions toward the 5E reports from principals working in different 
size districts. Principals from larger districts reported more favorable ratings than those from 
small or medium-sized districts. 


• The greatest differences were on their opinions toward the value of the 5E reports to 
the principals, in which 52% of those from large districts reported that the reports 
were valuable, while between 64% and 67% of those from small and medium 
districts disagreed with this statement. 

• Among the principals, 64% from large districts reported that the 5E reports were 
not valuable to teachers, compared to 77% and 80% from small and medium 
districts. 


Majority of school 
leaders rate use 
of 5E for planning 
unfavorably; however, 
principals reported 
more use of 5E for 
planning purposes 
than superintendents. 


Use of 5E Survey Data for Planning 

As was the case in the previous section, a majority of respondents either disagreed or strongly 
disagreed with the items pertaining to utilizing 5Essentials data for planning (see Figure 4). 
Both principals and superintendents provided somewhat more favorable responses regarding 
using the 5E results for planning and continuous improvement (33% for superintendents; 
48% for principals), as opposed to using the 5E data specifically in improvement plans either 
at the district (14%) or school (28%) level. 
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Mean 


District/School use 5E data for planning and/or 
continuous improvement. 


District/School modified District/School Improvement 
Plan based on 5E Survey data results. 


Superintendent (n=261) 
Principal (n=563) 


26 . 1 % 


16 . 2 % 


Superintendent (n=260) 
Principal (n=559) 


District/School used 5E data for other reasons. 


Superintendent (n=258) 
Principal (n=547) 



1 .5% 

2.08 

5.5% 

2.37 

0.8% 

1.83 

3.8% 

2.11 

0.4% 

1.99 

2.6% 

2.25 


0% 20% 40% 60% 80% 

i Strongly Disagree (1 ) ■ Disagree (2) Agree (3) ■ Strongly Agree (4) 


100% 


Figure 4. Use of 5Essentials Survey Data for Improvement Planning: Difference by Leadership Role 


Leadership Role Comparisons 

Principals provided statistically significantly more positive responses than superintendents to 
the use of the 5E data for planning purposes. Relatively more of the principals reported that 
they used the 5Essentials results for continuous improvement planning, specifically for school 
improvement plans, and for other reasons. 


Regional Comparisons 

There were not any statistically significant differences in superintendents’ reported use of 
the 5E data for planning purposes based on the region of the state. Elowever, there were 
statistically significant differences in whether principals reported using the 5E data for 
SIP purposes, with more Cook County principals using the data for school improvement 
planning (40%) than principals from other regions of the state (18% to 28%). There were 
not statistically significant differences in use of the data for continuous improvement or 
other reasons based on region, however only those principals from Cook County reported a 
favorable rating (over 50% agreement) on these items. 

Enrollment Size Comparisons: small (<600), medium (600-2,000), and 
large (>2,000) 

There were not any statistically significant differences in superintendents’ use of the 5E data 
for planning purposes based on the size of the district they were from. There was statistically 
reliable evidence that principals did use the 5E data differently for other reasons (besides 
continuous improvement planning and SIP) depending on the size of their district. In 
descriptive analyses of this sample of principals, principals from larger districts used the 5E 
data more than those from small or medium districts, 7 to 12 percentage points more. 


More Cook County 
principals reported 
using the 5E data 
for SIP purposes. 


Use of 5E Survey Data for Other Reasons 

Responses to items probing how respondents used the data, revealed some higher use 
activities and some lower use activities. Those activities that over 50% of the respondents 
endorsed included: reviewed/ discussed results at the district leadership level (82% for 
superintendents, 70% for principals), made efforts to ensure teachers received data (68% for 
superintendents, 80% for principals), and reviewed/discussed results with SIP teams (58% 
for superintendents, 69% for principals). Some of the lower use activities were: featured 
the data on the school website or in school communications (31% for superintendents, 


Principals and 
superintendents 
used the 5E data 
most often to discuss 
results with district 
leadership and SIP 
teams and to share 
results with teachers. 
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29% for principals), presented reports to parent or other community groups (11% for 
superintendents, 22% for principals), and discussed the data with parents (13% for 
superintendents, 21% for principals) (see Figure 5). Note, that percentages were calculated 
as the number endorsing ‘yes’ for the item divided by the total number of superintendents 
or principals that endorsed any of the utilization items. This method could overestimate 
the utilization, in that persons who did not endorse any of the items were not considered 
a responder for these calculations. However, because there was no way to distinguish non- 
responders and all ‘no’ answers, we could not more accurately estimate these percentages. 


Reviewed/discussed results at the district 
leadership level. 


Made efforts to ensure teachers have 
received the data. 


Reviewed/discussed results in school 
improvement teams. 


Presented the results at a school board meeting. 


Made efforts to ensure parents have 
received the data. 


Reviewed/discussed results in other teams. 


Featured the data on the district website or 
in district communications. 


Presented reports to parent (e.g., PTA) or other 
community groups. 


Discussed the data with parents of students 
at your school(s). 


Superintendent 

Principal 


Superintendent 

Principal 


Superintendent 

Principal 


Superintendent 

Principal 


Superintendent 

Principal 


Superintendent 

Principal 


Superintendent 

Principal 


Superintendent 

Principal 


Superintendent 

Principal 


Response 

Count 


(n=228) 

(n=502) 

(n=228) 

(n=502) 

(n=228) 

(n=502) 

(n=228) 

(n=502) 

(n=228) 

(n=502) 

(n=228) 

(n=502) 

(n=228) 

(n=502) 

(n=228) 

(n=502) 

(n=228) 

(n=502) 



187 

350 

155 

400 

131 

347 

134 

209 

95 

222 

72 

215 

70 

146 

25 

111 

29 

103 


0% 20% 40% 60% 80% 100% 


Figure 5. Use of 5Essentials Survey Data for Other Uses: Differences by Leadership Role. 

Note that percentages are calculated as a function of those that responded ‘yes’ to at least one use. 
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Leadership Role Comparisons 

There were several items in which there was a larger than 1 0 percentage point difference 
between superintendents and principals who endorsed utilization of the items. The greatest 
differences between superintendents and principals in this sample of respondents were on the 
following items: 

• Presented the results at a school board meeting (superintendents 17 percentage 
points higher) 

• Reviewed/ discussed results at the district leadership level (superintendents 12 
percentage points higher) 

• Made efforts to ensure that teachers received the data (principals 12 percentage 
points higher) 

• Reviewed/ discussed results with SIP teams (principals 1 1 percentage points higher) 

• Reviewed/ discussed results with other teams (principals 1 1 percentage points higher) 

• Presented reports to parents and other community groups (principals 1 1 percentage 
points higher). 

These differences appear to reflect the natural roles of each, i.e., superintendents were 
involved in more district-level use of the data and principals were involved in more building- 
level use of the data. 

Relationship Between Perceived Value of the 5Essentials Survey Data 
and Use of 5Essentials Report 

We explored the relationship between using the 5Essentials data for planning and/or 
continuous improvement and school improvement planning and how different components 
of the 5Essentials survey were valued by both principals and superintendents. Phi correlation 
coefficients were calculated for all correlations based on whether or not they agreed (strongly 
agree and agree were aggregated) or not (strongly disagree and disagree were aggregated) with 
each of the statements. Corrections for inflated Type I error were conducted with Benjamini 
and Elochberg’s (1995) false discovery rate. Additionally, the strength of the correlations 
were categorized as large (()) > .5), medium (.3 > ()) > .5) or small (()) < .1) for statistically 
significant correlations (Cohen, 1988; 1992). The correlations between using the 5Essentials 
data for continuous improvement with using the 5Essentials data to modify district/ 
school improvement plans were relatively strong (ij) = .50 for superintendents, ()> = .58 for 
principals). This indicated that both principals and superintendents who used information 
from the 5Essentials for continuous improvement purposes, were likely to also use it to 
modify school improvement plans. 


Principals and 
superintendents 
who used the 5E 
data for continuous 
improvement purposes 
were likely to also use 
it to modify school 
improvement plans. 
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The more school 
administrators 
who perceived the 
teacher survey data 
as useful, the more 
likely they used the 5E 
data for continuous 
improvement planning 
and modifying 
improvement plans. 


The more principals 
and superintendents 
agreed that the 
teacher survey 
addressed local 
interests, the more 
likely they used 5E 
data for improvement 
planning. 


Teacher Survey 

Five of the teacher survey items were consistently statistically significantly correlated 
with both 5E data use for planning and/or continuous improvement and for modifying 
district/school improvement plans, whether the respondent: found the teacher survey 
data valuable, believed that the communiiallftli^vfSs sufficient, believed that teachers were 
willing to participate in the survey, believed that the survey administration procedures 
were adequate to ensure fair, reliable data, and viewed the items to be of local interest (see 
Table 5). Agreeing that the information from the teacher survey was valuable had a strong, 
positive relationship with continuous improvement planning for superintendents and a 
moderate, positive relationship for principals, as well as a moderate, positive relationship 
with modifying district/school improvement plans for both superintendents and principals. 
Agreeing that the information from the teacher survey was of local interest had a moderate, 
positive relationship with use of the 5E data for district/school improvement planning for 
both superintendents and principals. The remaining significant correlations with the teacher 
survey items were small effects. 

Table 5. Correlations of Leader-Reported Use of 5E Data with Leaders’ Opinions about the Teacher 
Survey 



Continuous 

Improvement 

Modified School 
Improvement Plan 


Superintendents 

Principals 

Superintendents 

Principals 

Teacher survey generated data 1 found valuable 

0.52 

0.48 

0.33 

0.41 

Communication was sufficient to administer 
survey 

0.22 

0.27 

0.19 

0.22 

Teachers were willing to participate in survey 

0.24 

0.15 

0.15 

0.14 

Survey administration procedures were adequate 
to generate fair, reliable data 

0.29 

0.25 

0.23 

0.19 

Most of the survey questions adequately address 
issues of interest to my district/school 

0.36 

0.34 

0.21 

0.28 

District/school encountered significant logistical 
challenges due to technological issues 

ns 

ns 

ns 

ns 

District encountered significant challenges with 
survey administration for other reasons 

ns 

-0.12 

ns 

ns 


Large Effect Medium Effect Small Effect ns = Not Significant 


The more principals 
and superintendents 
agreed the 5E 
student survey was 
valuable, the more 
likely they used the 5E 
data for continuous 
improvement 
and modifying 
improvement plans. 


Student Survey 

Five of the student survey items were statistically significantly correlated with use of the 
5E data for planning and/or continuous improvement, whether the respondent: found the 
student survey data valuable, believed that the communication was sufficient, believed that 
students were willing to participate in the survey, believed that the survey administration 
procedures were adequate to ensure fair, reliable data, and viewed the items to be of local 
interest (see Table 6). Agreeing that the information from the student survey was valuable 
had a moderate, positive relationship with continuous improvement planning for both 
superintendents and principals. Agreeing that the information from the teacher survey was of 
local interest had a moderate, positive relationship with use of the 5E data for district/school 
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continuous improvement planning for superintendents. The remaining significant correlations 
with the teacher survey items were small effects. 

Four of the student survey items were statistically significantly correlated with use of the 
5E data for modifying district/school improvement plans for superintendents: whether the 
respondent found the student survey data valuable, believed that the communication was 
sufficient, believed that the survey administration procedures were adequate to ensure fair, 
reliable data, and viewed the items to be of local interest (see Table 6) and each of these were 
also significantly correlated with modifying district/school improvement plans for principals, 
except the item indicating communication was adequate. Agreeing that the information from 
the student survey was valuable had a moderate, positive relationship with modifying district/ 
school improvement plans for both superintendents and principals. The remaining significant 
correlations with the student survey items were small effects. 


Table 6. Correlations of Leader-Reported Use of 5E Data with Leaders’ Opinions about the Student 
Survey 



Continuous 

Improvement 

Modified School 
Improvement Plan 

M TH 

The more 
superintendents 
agreed the student 
survey addressed local 
interests, the more 
likely they used the 5E 


Superintendents 

Principals 

Superintendents 

Principals 

Student survey generated data 1 found valuable 

0.42 

0.42 

0.36 

0.32 

Communication was sufficient to administer 
survey 

0.28 

0.18 

0.21 

ns 

Students were willing to participate in survey 

0.17 

0.19 

ns 

ns 

data for continuous 
improvement. 

Survey administration procedures were adequate 
to generate fair, reliable data 

0.21 

0.29 

0.28 

0.20 


Most of the survey questions adequately address 
issues of interest to my district/school 

0.31 

0.28 

0.28 

0.29 


District/school encountered significant logistical 
challenges due to technological issues 

ns 

ns 

ns 

ns 


District encountered significant challenges with 
survey administration for other reasons 

ns 

ns 

ns 

ns 



Large Effect Medium Effect Small Effect ns = Not Significant 


5Essentials Reports 


District and school leaders’ views of the 5Essentials reports were predictive of whether they 
used the data for planning purposes. Both use of the 5E data for district/school continuous 
improvement planning and modifying district/school plans were statistically significantly related 
to all five items referring to their opinions of the reports: whether they viewed the reports as 
providing valuable information, providing new information, easy to use, whether they believed 
teachers found the reports valuable, and whether they believed parents found the reports 
valuable (see Table 7). Agreeing that the reports were easy to use and that teachers valued the 
reports had strong, positive relationships with use of the 5E data for continuous improvement 
planning for superintendents. The remaining report feedback items were moderately, positively 
correlated with district continuous improvement planning for superintendents. Agreeing 
that the reports were valuable had a large, positive correlation with continuous improvement 


The more 
superintendents 
agreed the 5E reports 
were easy to use and 
that teachers valued 
the report, the more 
likely they used the 5E 
data for continuous 
improvement. 
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The more principals 
agreed that the reports 
provided valuable 
information, the more 
likely they used the 5E 
data for continuous 
improvement. 


planning for principals, whereas the remaining report feedback items were moderately, 
positively correlated with school continuous improvement planning for principals. Views of 
all of the report feedback items were moderately, positively correlated with modifying district/ 
school improvement plans for both superintendents and principals. 


Table 7. Correlations of Leader-Reported Use of 5E Data with Leaders’ Opinions about the Survey 
Reports 



Continuous 

Improvement 

Modified School 
Improvement Plan 


Superintendents 

Principals 

Superintendents 

Principals 

Survey reports provided valuable information in 
my role as superintendent/principal 

0.48 

0.51 

0.45 

0.37 

Survey reports provided new information in my 
role as superintendent/principal 

0.34 

0.44 

0.35 

0.36 

Survey reports provided easy to use information 
in my role as superintendent/principal 

0.51 

0.44 

0.40 

0.36 

Teachers in my district/school found the survey 
reports valuable 

0.51 

0.45 

0.43 

0.43 

Parents in my district/school found the survey 
reports valuable 

0.35 

0.41 

0.36 

0.39 


Large Effect Medium Effect 


36 IERC 2014-2 


Comparison of District Interviews with ISBE Survey Responses 


Next, we compared this study’s interview data with survey responses and comments about 
the 5E Survey from the January 2014 ISBE statewide survey with superintendents and 
principals to gauge the representativeness of the information obtained from our participating 
15 districts. We found similarities across the interview and survey data related to use of the 
5E data, perceived value of the 5E data and reports, and reasons for not using the 5E data. 

Both survey respondents and those interviewed in district site visits indicated most often 
sharing the 5E data with district leadership, teachers, and school improvement teams, and 
to a lesser extent with school boards and parents. A small percentage of survey respondents 
(approximately 30%) and only a couple of site visit districts shared the 5E data through their 
districts’ website. More superintendents and principals from the survey disagreed/strongly 
disagreed with the item regarding use of the 5E data for continuous improvement (slightly 
more favorable) or to modify their school improvement plan. This corresponds with the 
interviews where we heard from superintendents, principals, and SIP members that there was 
limited use of the 5E results, and that it was typically for general improvement planning but 
not specifically used to modify the district/school improvement plan. 

Regarding the perceived value of the 5E data and results, we again saw parallels between the 
survey data and the interviews. The survey data indicated moderate to strong relationships 
between the perceived value and local interest of the teacher and student survey items and 
the use the data for planning purposes. From the interviews, some participants remarked the 
5E data were useful or valuable because it provided multiple perspectives and informed their 
improvement planning; however, most participants were not overwhelmingly positive due to 
concerns about the credibility and usefulness of the data. 

Most of the survey comments, as well as what we heard from district participants, were 
concerns they had about the 5E Survey and data. District participants’ reasons for not 
utilizing the 5E data were very much in line with the survey comments. Similar to the 
district interviews, survey participants cited issues with data credibility most often for not 
utilizing the 5E data. According to survey respondents, lack of credibility stemmed primarily 
from doubts about valid respondents, problems with survey language, and lack of buy-in. 
These issues were raised by both superintendents and principals in the online surveys. 

Survey respondents also cited lack of a secure login as another major concern. These 
individuals were worried that multiple entries by the same individual and responses from 
individuals not affiliated with their school could result in lack of integrity in the 5E data. 
Comments from the survey included: 

“Inaccurate accounting of teachers in the district resulted in misreporting of percent of teachers 
participating. ” 

“Students could take the survey as many times as they wanted, for any school in the district 
they wanted. ” 


We found similarities 
between responses 
from district interviews 
and survey data: 

• 5E data shared, 
but limited use 
for improvement 
planning; 

• the more perceived 
value and 
connection to local 
interest, the more 
likely to use for 
planning purposes; 
and, 


concerns about the 
credibility of the 5E 
data limited use. 
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Concerns about 
teachers not 
completing the survey 
objectively were 
expressed more often 
in survey comments. 


Similar to interview participants from our 15 districts, numerous survey respondents were 
also concerned about problems surrounding the survey language. Our analysis of the 
open-ended survey comments revealed worries about confusing questions, vague terms, bias 
toward Chicago Public Schools model, and the lack of “does not apply” option from both the 
superintendent and principal respondents. Examples of survey commented included: 

“The language of the questions was challenging for many middle school children to 
comprehend. What is a “rigorous curriculum” to a 6th grader ?” 

“Some questions were not applicable and there were not options to not respond. ” 

“The questions simply do not align with our administrative structure, as Assistant Principals 
carry on many of the Principal functions. The same could be said for Department Chairs. 
Teachers were VERY confused about who they were even responding to in given questions. ” 

Lack of buy-in from administrators, teachers, parents, and students was also evident in the 
survey responses. Our analysis of the open-ended survey comments responses revealed that 
some principals and superintendents discounted the 5E results due to concerns surrounding 
the research supporting the 5Essentials, the seriousness with which respondents replied to 
the survey, and the appropriateness of using student data. Also, several survey respondents 
expressed doubt that a survey based on Chicago Public Schools could apply to all other 
districts in the state. Survey comments included: 

“In my opinion, kids might not always be the most reliable sources of information. As a 
middle school principal, I have plenty of data to prove this. ” 

“Students, like teachers, never embraced the importance of collecting this data. ” 

“The established “cut scores” are highly suspect. I do not believe the consolidation of sub- 
components into a single rating is fair or reliable. ” 

“There were no good uses for the data as the questions were biased for the CPS school system. 
The demographics for which those were created did not match the demographics where they 
were being taken outside of CPS. We are not a one size fits all. ” 

One major difference between our interview data and the survey data was that a much larger 
number of survey comments related to principals’ and superintendents’ concerns about 
teachers not completing the survey objectively. Many survey respondents believed some 
teachers were overly and unfairly critical of their administrators’ performance. Similar to 
the interview data, survey respondents indicated the harsh criticism may have been due 
to survey timing coinciding with teacher evaluations, reductions-in-force, and significant 
administrative decisions within the district for school improvement purposes. This larger 
number of responses from the ISBE survey may have been due to the anonymous nature of 
the survey. Survey comments included: 

“We are a small school. Negative responses from one or two staff members had a huge effect 
on the results. Teacher morale was at an all-time low when they looked at the rating system. 
Increased mistrust among the staff.” 

“Administering this during the same time period when we are required to notify teachers that 
their jobs are being cut is inherently unfair and skews the data. ” 
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“This information is very bias[ed] and several teachers use it as means to ‘get back at the 
principal’ for events that have taken place that they do not agree with. ” 

Other reasons cited to a lesser extent included: being overwhelmed by too many other 
initiatives, concern about impact on jobs, problems with technology, and districts/schools 
already preferring another climate survey. Examples from survey comments included: 

“We already have to do a number of surveys throughout the year for other state mandates. This 
one was just more on top of more. ” 

“The ratings of very strong to very weak, unfortunately, come across as evaluative. This is 
incredibly problematic. ” 

“Providing the opportunity and sufficient bandwidth for students to work to complete the 
survey when computers froze and students couldn’t continue was a challenge as well. ” 

Similar to the district interviews, a small number of individuals from the ISBE survey 
remarked that the 5E survey was useful, although typically with a caveat. In general, the 5E 
data was seen as valuable because it sparked conversations or provided a different perspective. 
However, caveats included that the data should not be made public or did not provide any 
new information: 

“The overall [student] data was valuable and generated in depth conversations, but should be 
an internal document to be shared with teachers. ” 

“As a whole the survey has definitely given us a big picture perspective on our building. ” 

“I think the intent of the survey is good, but at this point it is just seen as one more hoop to 
jump through in a growing list of tasks assigned by the state. ” 

“The survey was useful, we find all surveys useful, but we didn’t learn any new information. ” 

In summary, the responses from the statewide ISBE survey correspond closely to the 
information gathered at the district interviews. Issues related to the credibility of the data 
were raised most often by the superintendents and principals completing the survey, as 
well as most of the participants in the district interviews. The statewide survey allows for 
wider representation of the results across the state, while the interview data provide a richer 
description of similar results. 



Similar to the district 
interviews, a small 
number of individuals 
from the ISBE survey 
remarked that the 5E 
survey was useful, 
although typically 
with a caveat. 


Level of Data Supports for Data-lnformed Decision Making 

In addition to examining how Illinois districts are using the 5Essentials Survey data for 
school improvement, participants from our sample were also asked about their access to data, 
supports for data use, and data-informed decision making practices. Using the conceptual 
framework developed by Means, et al. (2009), participants were asked to assess the extent 
to which their schools were meeting the prerequisites and supporting conditions for data- 
informed decision making. 

Means et al. (2009) identified six major prerequisites and supports for data-informed 
decision making: “(a) state, district and school data systems; (b) leadership for educational 
improvement and the use of data; (c) tools for generating actionable data; (d) social structures 
and time set aside for analyzing and interpreting data; (e) professional development and 
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technical support for data interpretation; and (f) tools for acting on data” (p. 3). Participants 
were asked to respond to a series of statements based on probes used by Means et al. (2009) 
to determine if the necessary conditions were being met to a great extent, modest extent, 
some extent, or not at all. They were also asked to provide examples to demonstrate the basis 
for their assessments. See Appendix A for the interview protocol and questions. 

Table 8. Level of Data Supports for Participating Districts 


District 

Locale 

Size 

Type 

Data 

Systems 

Access 

Leadership 

for 

Improvement 

Tools for 
Generating 
Data 

Social 

Structures 

Professional 
Development 
& Tech 
Support 

Tools for 
Acting on 
Data 

District 

Score 

1 

City, Mid 

L 

P-12 

M 

M 

S 

G 

M 

M 

M 

2 

City, Small 

L 

P-8 

G 

G 

G 

G 

G 

G 

G 

3 

City, Small 

L 

P-12 

G 

M 

G 

G 

S 

S 

M+ 

4 

Suburban, Large 

L 

9-12 

M 

M 

S 

M 

S 

-M 

-M 

5 

Suburban, Large 

L 

P-12 

G 

M+ 

G 

M+ 

M+ 

M 

M+ 

6 

Suburban, Large 

L 

P-12 

G 

S 

S 

M 

M 

-M 

-M 

7 

Suburban, Large 

L 

P-8 

G 

M 

M 

S 

S 

M 

M 

8 

Town, Fringe 

M 

P-12 

G 

-M 

M 

S 

S 

S 

S+ 

9 

Town, Distant 

M 

9-12 

G 

M+ 

S 

M 

-M 

S 

-M 

10 

Town, Remote 

M 

P-8 

G 

M 

— 

S 

-M 

M+ 

M 

11 

Town, Remote 

L 

P-12 

G 

M 

M 

S 

S 

S 

-M 

12 

Rural, Fringe 

L 

P-12 

G 

S 

M 

S 

-M 

M 

-M 

13 

Rural, Fringe 

S 

P-12 

G 

M 

M 

G 

-M 

M 

M 

14 

Rural, Distant 

S 

P-12 

M 

S 

S 

S 

S 

S 

S 

15 

Rural, Distant 

S 

K-8 

M 

S 

S 

-S 

-S 

S 

S 


Notes: This data is arranged based on the locale code assigned to the school district by the National Center for Education Statistics. 

Codes indicate the extent to which each support was present during the 2013-2014 school year based on interviews with district/school personnel: 

-S = not at all to some extent; S = to some extent; S+ = to more than some extent; 

-M = to a less than modest extent; M = to a modest extent; M+ = to a more than modest extent; 

G = to a great extent. 

An extended scale is used to report these findings because respondents frequently expressed gradations beyond the four choices (Great Extent, Modest Extent, 
Some Extent, or Not at All) given to them regarding the extent to which their district was moving towards implementation of a data-informed decision making culture. 


With only three 
exceptions , the overall 
assessments indicate 
that all of districts in 
our sample are at least 
approaching modest 
presence of supports 
for data-informed 
decision making. 


A compilation of responses from the 15 participating school districts is provided in Table 8. 
With only three exceptions, the overall assessments indicate that all districts in our sample 
are at least approaching modest presence of supports for data-informed decision making. 
However, only one district was rated as having supports available to a great extent (see 
far right column). For the most part, in the words of one of the superintendent’s, all of 
these districts are “in transition to data-driven decision making.” One of the lowest rated 
districts had recently experienced turnover in the superintendency, and the other two lowest 
rated districts were small, rural districts with limited resources (one of which had a great 
appreciation for data-informed decision making but lacked the support they required). The 
other district had mixed viewpoints. Some of the participants were interested in utilizing 
data but lacked access, though they anticipated using the 5E data in the future. On the other 
hand, one or two others indicated less need for data due to the small size of the district. 

It is interesting to note that two of the three school districts with the highest levels of 
supports for data-informed decision making (and the only small, rural school district with 
modest levels) were also current recipients of School Improvement Grants (SIG) and/or, 

Race to the Top funds. These respondents expressed much concern about the sustainability 
of their efforts without the grant support. These findings are very consistent with those of 
Means et al. (2009) regarding the role of federal programs to support implementation of 
data-informed decision making. Several respondents indicated that the loss of grant funds 
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would result in the loss of coaches and/or compensation for those on the “data team” who 
were most instrumental in assisting classroom teachers with connecting student data to 
improved instruction. 

Also in line with the findings of Means et al. (2009), we found that principals and teachers 
generally reported having access to all or most data for students in their schools, or at least on 
the students in their classrooms. While the specific data elements and the ease with which 
they could be obtained varied, 1 1 of the 1 5 sites indicated that data system access was great 
and the remaining 4 sites indicated a modest amount. As were the findings in the national 
study, few of the schools had integrated data systems designed to support school-level 
instructional decision making. One of the superintendents summed up the situation saying: 

“In our district they’re used to it. We don’t have a data warehouse, we don’t have a 
clearinghouse, we don’t have a dashboard. This is what we’re used to, in part because we 
have no technology budget. We’re used to having multiple logins, pulling data from separate 
places, and bringing them together to the table. Sometimes there’ll be three people sitting 
around the table, each has a different data system pulled up as they’re talking as a data team. 
Other times, we literally print off and we bring out and we spread out and we’re literally 
highlighting on paper. ” 

However, another superintendent spoke about promising developments to come from 
IlliniCloud, a consortium of local school districts intended to provide state of the art 
computing resources for their member districts. Collaborative efforts like this offer 
opportunities for both large and small districts to meet their data analysis needs. The 
superintendent said, 

“Other states have done it through the state government that no one trusts. Or a company 
comes in and all they care about is profits. And so the IlliniCloud has a model that I think is 
actually going to work because I can talk to other superintendents and we’re not going to share 
our data with any of these companies. ” 

Access to such resources is likely to have a positive impact on district and school leadership 
for instructional improvement regarding data use. Currently, a third of the participating 
districts found leadership in this area to be less than modest (-M and below), while just 
a fifth assessed their leadership to be more than modest (M+ and above). Respondents 
used terms like “emerging” and “moving” to describe their leadership efforts to provide the 
necessary guidance and support for data-informed decision making. It appears that there 
is need for greater alignment between assessment data, the district curriculum, and state 
standards to support instructional practice. These findings, too, are quite in line with what 
Means et al. (2009) found. 

The support that rated lowest was the provision of professional development and technical 
support for data interpretation. In order to address these professional development needs, 
some districts have pooled resources to form consortia like the IlliniCloud and the Heart of 
Illinois to make the training more accessible and relevant to the needs of the local schools. 

In terms of technical support, respondents frequently spoke about one or two people being 
available to address the needs of the entire district, or relying on math or science teachers 
to provide data analysis skills for school teams. Even those schools that have numeracy 
and/or literacy coaches to provide this support, expressed concern that these positions are 



Similar to the national 
study (Means et al., 
2009), we found 
that principals and 
teachers generally 
reported having 
access to all or most 
data for students in 
their schools, or at 
least on the students 
in their classrooms. 


Professional 
development and 
technical support for 
data interpretation 
was rated lowest in 
terms of available 
data supports. 
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Use of the Illinois 5Essentials Survey Data 



The majority of 
districts in our sample 
are making great 
strides to establish 
cultures promoting 
data-informed 
decision. Yet they 
face barriers to 
access data experts, 
tools, and training. 


Levels of data support 
and use of 5E data 
are slightly related. 


grant dependent and limited in what they could provide to teachers outside of the grant 
parameters. These results are similar to the national study in which “making data analysis 
experts (sometimes called data coaches) available to school staff is one of the least common 
supports,” while technical expertise in systems, networks, or databases is provided at many 
school districts (Means, Padilla, & Gallagher, 2010, p. xiii). 

Overall, the vast majority of participating school districts are making great strides to establish 
cultures that promote data-informed decision making. Yet, they face barriers regarding 
access to data experts, tools, and meaningful training to make this shift. Locale appears to 
influence the presence of the prerequisites and supporting conditions for data-informed 
decision making more than size or the type of the district, and these barriers seem to be 
most pronounced in rural areas and more remote towns. This may be due to the fact that 
the district’s locale often reflects its tax base and the financial resources available for schools. 
However, efforts to improve financial and human resources are a positive sign of the growing 
commitment among Illinois’ school districts to make the changes required to improve 
student achievement through the use of data-informed decision making. 

Comparing Use of 5Essentials Data with Levels of Data Supports 

We next examined whether districts’ levels of supports for data use were related to their 
use of the 5Essentials data. Based on their own descriptions of their use of the 5E data 
(see Table 2), we categorized the districts into four groups: 1) Share/Review, Analyze and 
Implement Action; 2) Share/Review and Identify; 3) Share Data and Plans for Future Use; 
and 4) Limited Share Data. We also collapsed the level of support for the district grade 
(S-G) (Table 8) into three levels: (1) “not at all” to “some extent” (-S, S); (2) “more than 
some” to “modest extent” (S+, -M, M); and (3) “more than modest” to “great extent” (M+, 
G). We then compared the four groups based on 5E data use with the districts’ scores on the 
availability of data supports. Figure 6 graphically shows the relationship between 5E data 
use and level of data supports, based on interview data with the district participants. 3 

As seen in the figure, levels of data support and use of the 5E data are only slightly related. 
Three of the 15 districts would be considered advanced 5E data users (Group 1); however, 
they have modest levels of data supports for data-informed decision making. These Group 
1 districts shared, reviewed, analyzed, and implemented new programs based on their 5E 
results. On the other hand, the Group 4 districts reported limited use (e.g., shared 5E data 
only) despite having modest to great levels of data supports. One participant described it 
this way: 

“I don’t know if there’s a policy that says you have to do this, but it’s just sort of the culture of 
this place where we do a lot of ..we have a board that’s very data conscious and so our district 
goals that the board has worked with us on are around achievement, three of the six are 
around achievement. So, the culture of this place has been use data to make decisions about 
what you’re doing in the classroom. ’ So that’s not a foreign concept here. It’s just trying to 
build that in and keep people current on what are we using. What assumptions can you make 
with it? How can you interpret it? That takes a lot of PD. But we do what we can in the 


These analyses are based only on available interview data with district participants. We did not observe or 
obtain corroborating evidence. Thus, some districts may over- or under-estimate their supports. Nevertheless, 
we feel that these analyses help us to obtain a general sense of participants’ perceptions of the availability of data 
supports compared to their districts’ use of the 5E data. 
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Use of 5E Data 
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Note: Numbers within the chart are district numbers, in reference to Table 2. 

To read the chart: District 1,4,9 had less than Modest (-M) to Modest level of supports for data 
informed decision making and these Districts had the most utilization of the 5E data by sharing, 
reviewing, analyzing, and implementing a program based on the 5E results. 

Figure 6. Use of 5Essentials Data by Level of Data Supports 


midst of a lot of other things that we’re supposed to be doing. ” 

Interestingly, both the Group 1 and Group 4 districts expressed concern about 
the credibility of the data, although Group 4 districts voiced stronger objections. 

However, Group 1 districts seemed more willing to work with the 5E data and 
pull out what they found useful, whereas the Group 4 districts discounted all of 
the results and lacked buy-in to the 5E Survey in general. One interview with two 
participants commented on this issue of trust and credibility of the data: 

“I think it [5E Survey] will fit in when the trust is established. So it’s not a bad 
survey, we have to get through the drama and the trust issue. That’s with any type of 
data driven decisions; you’ve got to get through the human emotion before you get to 
the real one. ” 

“I think until the survey itself works out how it’s measuring and how it’s comparing, you’re 
also not going to get over that, too. ” 

Group 2 districts reported modest data supports for decision making and had similar uses 
of the 5E data. These districts typically shared their data with more individuals within the 
district/school (i.e., principals, teachers, school improvement teams), as well as with a wider 
audience including the school board, parents, and the community. Reviewing the 5E data 
with the SIP teams resulted in identifying strengths, areas for improvement, goals, and 
professional development areas. Group 1 district completed most of these activities, as well; 
however, Group 1 districts took the next step and implemented changes based on the 5E 
data. Lastly, Group 3 districts spanned the continuum for the availability of data supports. 
All of them shared their data with principals and teachers and were willing to consider future 


District 

Locale 

Size 

Type 

1 

City, Mid 

L 

P-12 

2 

City, Small 

L 

P-8 

3 

City, Small 

L 

P-12 

4 

Suburban, Large 

L 

9-12 

5 

Suburban, Large 

L 

P-12 

6 

Suburban, Large 

L 

P-12 

7 

Suburban, Large 

L 

P-8 

8 

Town, Fringe 

M 

P-12 

9 

Town, Distant 

M 

9-12 

10 

Town, Remote 

M 

P-8 

11 

Town, Remote 

L 

P-12 

12 

Rural, Fringe 

L 

P-12 

13 

Rural, Fringe 

S 

P-12 

14 

Rural, Distant 

S 

P-12 

15 

Rural, Distant 

s 

K-8 
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use of the 5E data if validity issues concerning the 5E data were addressed. The difference 
in levels of data supports for Group 3 districts is likely due to the size of the district and 
availability of external resources (e.g., Race to the Top, SIG grants). 

Thus, we found levels of data support and use of the 5E data were slightly related, indicating 
other factors are also contributing to districts’ decisions to utilize the 5E data. In the 1 5 
participating districts we found districts with modest to great supports on both ends of the 
continuum for utilizing the 5E data. We also found that districts (Groups 1 and 2) utilizing 
the 5E data beyond just sharing the results require at least modest levels of data supports. 
Moving forward, shifting Group 3 district to a higher level of use will require potentially 
more supports for the smaller districts; and, shifting Groups 3 and 4 will require alleviating 
strong concerns about the validity of the data. One participant expressed it this way, 
“Somehow, you’ve got to get your organization, your political organization leaders to buy 
into it and trust.” 



Recent Activities and Policy Changes Impacting 5Essentials Survey 


ISBE and UChicago Impact have begun to address many of the concerns raised by interview participants and survey 
respondents. Listed below are recent policy changes or activities to improve the 5Essentials Survey and increase 
utilization of the results. 

1. In Spring 2014, ISBE and UChicago Impact conducted focus groups with administrators and teachers 
around the state to solicit feedback for problematic language or context. In addition, pilot testing of 
alternative language for survey items took place during the 2014 survey administration. For details on the 
results and recommendations based on the focus groups, see http://help.ccsrsurvey.uchicago.edu/customer/ 
portal/articles/1461 l43-illinois-5essentials-focus-group-report. In addition, based on feedback from the 
focus groups, the color scheme on the 5E reports will change from greed/red to shades of blue and wording 
will change from strong/weak to more/less implementation. 

2. A pilot study utilizing rosters for survey participation was conducted in Spring 2014 to examine issues 
with multiple entries and invalid respondents. Results identified the need for quality assurance of the 
roster information, the inclusion of assistant administrators (and others) who support the implementation 
process, and clearer communication/understanding about the use of students’ state ID numbers (rather than 
their school IDs) and the option of a provisional login. Based on these results, ISBE, in consultation with 
UChicago Impact, has decided to roster all student surveys and extend the rostering pilot for the teacher 
survey to 75 districts across the state for the 5Essentials Survey Spring 2015 administration (Peter Godard, 
personal communication, August 28, 2014). For more details about the pilot study, see http://www.isbe. 
net/5essentials/pdf/5E-roster-survey-rptl4.pdf 

3. Public Act 098-0648 was recently modified to allow school districts to elect to use an alternate survey 
which is required to be pre-approved by ISBE. ISBE’s process to identify 2-3 alternative surveys is currently 
underway, in consultation with a committee of education stakeholders, with final instrument selection 
expected by late Fall 2014. The statute (105 ILCS 5/2-3.153) requires the alternative surveys meet the 
following criteria: 

a. provide summation of indicators to be included on the school report card 

b. be able to provide summary reports for each district and attendance center intended for parents and 
community stakeholders 

c. meet scale reliability requirements using accepted testing measures 

d. provide research-based evidence linking instrument contents to one or more improved student outcomes 

e. have undergone and documented testing to prove validity 

4. According to the July 21, 2014 Weekly Message from State Superintendent Christopher Koch, the 
administration window for the spring 20 1 5 5E Survey will be January 12, 2015 - March 13,2015. While 
this change does not reduce the number of mandates, it does take into consideration concerns about the 5E 
Survey administration occurring during other sensitive times during spring. In addition, 2013 and 2014 
data will be normed (re-benchmarked) to the scores to the 2013 statewide data with similar schools based 
on grade level configuration, type of school setting, and percent of students qualifying for free and reduced 
lunch. 

5. 5E Surveys for younger students are currently in development and will be piloted soon. 
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Use of the Illinois 5Essentials Survey Data 



Use of the 5E results 
is still in early stages 
of adoption for many 
districts in our sample. 


The primary purpose of this study was to examine how Illinois school districts are utilizing 
the Illinois 5Essentials Survey and its results, particularly for school improvement, and 
to solicit suggestions to further improve the implementation of the 5E Survey. Based 
on interview data and survey data, we learned that some stakeholders found the 5E data 
valuable in that it added additional perspectives and informed their school planning 
activities, although this was not the majority of the participants. We also found that use of 
the 5Essentials results is still in early stages of adoption for many districts. Most districts 
reviewed the 5E results with principals, teachers, and school improvement teams, but usage 
was primarily limited to general discussions about districts’ strengths and weaknesses. Some 
districts and schools have begun to integrate the 5E results with their Rising Star school 
improvement process by aligning the 5Essentials with their improvement indicators and 
using the 5E results as evidence for the indicators. A few districts that are more advanced 
users of the 5E data have used the data to drive implementation of new programs or 
expansion of existing programs. In one setting, district office personnel with expertise 
to serve as data coaches worked with teachers to understand the 5E data and district 
administrators are considering incorporating 5E results as evidence in principal evaluations. 
Additionally, the survey data indicated moderate to strong relationships between the 
perceived value and local interest of the teacher and student survey items and the use of 
the5E data for planning purposes. 


Use of the 5E data 
was hindered for many 
reasons including 
concerns about the 
data, access to other 
climate surveys, and 
participants feeling 
overwhelmed. 


M 




Some districts chose 
to work with the 5E 
data, despite their 
concerns, while 
others did not. 




Respondents noted many reasons for not utilizing the 5E Survey and its results. Concerns 
about the credibility of the data created major barriers for many districts to utilize the 
5E results (from the original 2013 data release). Districts did not trust the data due to 
concerns about the validity and accuracy of responses (e.g., multiple entries and confusing 
survey language). Many interview participants and survey respondents felt that the survey 
did not meet local needs and the results were inappropriately normed against Chicago 
Public Schools, resulting in decreased buy-in. Districts that previously administered a local 
school climate survey or had access to similar data were less inclined to use the 5E Survey, 
particularly if they had concerns about the credibility of the 5E data. In addition, many 
participants reported feeling overwhelmed by the large number of major initiatives (e.g., 
PARCC, Common Core, new evaluation systems) currently being implemented statewide 
and around the same timeframe, and worried that these additional mandates may have 
overstretched available resources and created additional hurdles for implementation of the 
5E Survey. Some districts chose to work with the 5E data and glean useful information 
for their purposes despite their concerns. Elowever, about half of the participating districts 
and many survey respondents used the 5E data on a limited basis or not at all due to these 
strong concerns. It is worth noting, however, that many of these concerns are beginning 
to be addressed by recent ISBE activities and policy changes (e.g., soliciting feedback on 
problematic survey language, use of ISBE-approved alternative survey, norming based on 
2013 statewide data, earlier and longer 2015 survey administration, rostered student survey, 
expanded pilot of rostered teacher survey). 
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In addition to addressing the concerns described above, participants provided additional 
recommendations for improving the 5Essentials Survey and the usefulness of its results. 
Marketing the survey in order to motivate participation, along with increasing the breadth 
of participation and response rates, particularly for parents, were seen as viable strategies 
to increase buy-in. The strong to moderate correlations between perceived value and local 
interest of the 5E with use of the data for planning purposes also highlights the need to 
market the value of the survey data. To encourage utilization, participants recommended 
providing more immediate feedback and, more easily understood results (e.g., charts and 
graphs), as well as raw data files so that districts can disaggregate responses within schools to 
answer their data questions. Although some participants requested that the data be presented 
in context (normed scores versus frequencies), there were others who recommended against 
the use of color to represent relatively strong or weak results. Several participants suggested 
that results (particularly normed scores) should remain private to districts/schools, in order to 
avoid shifting the focus from productive school improvement planning to negative reactions 
by the public. Last, many participants requested more actionable feedback and concrete 
advice for next steps about how to improve weaknesses identified by the 5E Survey results. 

Many participants indicated that more in-depth training is needed for increased buy-in and 
utilization of the 5E data. Many participants, particularly teachers on SIP teams, were not 
familiar with even the basic information about the 5Essentials and research supporting its 
use. Some participants suggested increased awareness and understanding of the beneficial 
information from the 5E survey results would facilitate buy-in to complete the survey and 
to increase interest in making use of the data. Participants recommended offering training 
in both webinar and face-to-face formats. Webinars provided convenience and less expense, 
while face-to-face training provided opportunities to network and interact with other districts 
to learn about successful practices. Additional training in data analysis and application of the 
results was also recommended to maximize the usefulness of the 5E results to inform school 
improvement planning. Several participants commented on the effective implementation 
and training for the Rising Star school improvement process. Building on this success, along 
with networking with other advanced 5E data users may increase the utilization of the 5E 
data. 

There were some notable demographic differences in opinions toward the 5E data, reports, 
and reported use. First, survey responses indicated more buy-in to the 5Essentials from 
building principals than district superintendents. Principals had higher ratings than 
superintendents on five of the seven items regarding the teacher survey and had higher 
opinions of the 5E reports. Principals also reported more use of the 5E data for planning 
purposes than superintendents. Principals and superintendents reported using the data for 
different purposes that largely fit their leadership roles (e.g., superintendents highest reported 
use was reviewing/ discussing results at the district leadership level (82%) while principals’ 
highest reported use of the data was ensuring that teachers received the data (80%)). Second, 
there were some consistent differences in opinions about the 5E data between principals of 
different sized districts. Principals from large districts had more favorable views regarding: the 
value and local interest of the teacher survey items, whether the student survey items were 
fair and reliable and whether the survey items were of local interest, and the value of the 5E 
reports, while principals from moderate-sized districts had the least favorable views on these 
items. 


ISBE and UChicago 
Impact have begun 
to address many 
of the concerns 
raised by interview 
participants and 
survey respondents. 


More in-depth 
training is needed for 
increased buy-in and 
utilization of the 
5E data. 


w 


Principals reported 
more buy-in than 
superintendents 
in terms of their 
opinions toward the 
5E data, reports, 
and reported use. 


In general, principals 
from large districts 
had the most 
favorable views. 
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Use of the Illinois 5Essentials Survey Data 



Availability of supports 
for data use is only 
slightly related to 
districts’ use of the 
5E data. 


With the initial efforts 
to address concerns 
and Illinois school 
districts’ interest in 
measuring school 
climate and culture, 
increased utilization 
of the 5Essentials 
Survey and data 
looks promising. 


We found only a slight relationship between levels of data support and use of the 5E 
data. We found districts with modest to great levels of support on both ends of the usage 
continuum, suggesting that other factors moderate 5E data use. Our study suggests that 
these other factors likely include the availability of alternative school climate measures and 
strong concerns about data validity. It does appear that at least a moderate level of support 
is needed to integrate the 5E data in school improvement planning processes. For smaller 
districts with fewer financial and personnel resources, supports from external funding 
(e.g., Race to the Top, or SIG funding) or external entities (e.g., ISBE, regional offices of 
education, professional associations) may be needed to increase the likelihood of using 
the 5E data. As mentioned above, training was identified by many participants as a much 
needed support to be added to school districts. In addition, district leadership to encourage 
and facilitate data use may be an effective strategy to increase use of the 5E data. We learned 
from the interviews that familiarity and use of the 5E data clustered within districts. Thus, 
promoting use of the 5E and inspiring district leadership to create a school/district culture 
receptive to data use could expand the utilization of the 5E data. Working with these 
building leaders more directly and through their networks of professional associations might 
also facilitate 5E data use for school improvement. 

To conclude, utilization of the 5Essentials Survey and results varied widely across the 1 5 
participating districts in this study, ranging from districts that have implemented programs 
to districts refusing to use the 5E data due to data validity issues and other concerns. 
Statewide, based on the survey results, utilization is hampered due to similar data quality 
concerns. Steps to address these issues are currently underway, which will likely increase 
buy-in in the future. Additional strategies such as training or external resources to increase 
districts’ capacity to analyze and apply the 5E results to school improvement planning may 
be needed in districts with fewer data supports and emerging data-based cultures. A positive 
environment in which districts are interested in collecting school culture and climate data 
from their stakeholder to improve their schools currently exists in Illinois. The ability to 
capitalize and cultivate these positive attitudes to increase utilization of the 5Essentisals data, 
or other climate survey data, will be greatly influenced by addressing the concerns identified 
by its stakeholders. 
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Appendix A: Interview Protocol 


IERC Implementation Review of 5Essentials Survey 
Site Visit Interview Protocol - Spring 2014 
All Questions 

Background Information 

1 . Tell me a little about yourself, such as .... 

a. Your current position and any previous positions in the school/district? 

b. How long at this school/district? 

c. Also, for SIP committee interviews: What is your role on the school improvement committee? Member, 
process manager, etc? 

5Essentials (5E) Survey - Familiarity and Training 

2. How familiar would you say you are with the 5Essentials Survey and your district’s/school’s 5Essentials data? 

a. Did you receive any training or professional development related to the 5Essentiais Survey? Please 
explain. 

3. To what extent do you think others in your school/district understand how to analyze and interpret the 5Essentials 
school data? 

a. Has any training or other activities occurred to promote their understanding? If yes, please explain. 

4. If training in the use of the 5E data report were available, . . . 

a. What topics of training would you find most helpful? 

b. Who else at your district/school could benefit from 5E data training? (other teachers, Curriculum 
Director, social worker/counselors, etc) 

c. What would be your preferred option in which to receive training? (face-to-face, webinar, handouts, 
presentations at professional association meetings, district meetings, regional meetings, or other) 

School Improvement Planning Process & Use of 5Essentials data 

5. Please describe your district’s (school’s) improvement planning process? 

a. District Level OR School Level 

i. Who is involved? 

ii. How often do you meet? 

iii. Do you have a general process? 

iv. What types of activities or topics discussed? 

b. Do you participate in ISBE’s Rising Star planning process? If yes, please briefly explain. 

c. LISTEN FOR OR ASK IF PROMPTS ARE NEEDED: 

i. Establish school wide data team that sets tone for ongoing data use 

ii. Define critical teaching and learning concepts 

iii. Develop a written plan that articulates activities, roles, and responsibilities 

iv. Provide ongoing data leadership 

6. What data resources do you use in your school improvement planning process? 

7. Does your district/school use or plan to use the 5Essentials data? 

IF YES: ASK QUESTIONS #8 - #12 
IF PLAN TO USE: ASK QUESTIONS #13 - #15 
IF NO: ASK QUESTIONS #16- #17 
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IF YOU HAVE USED 5 Essentials data: 


8. Have you used the 5E data for school improvement planning? 

a. How do you most often use the 5E data for school improvement planning? 

b. What are some specific examples of how you utilized your 5E reports for your school improvement 
planning? 

c. PROMPTS IF NEEDED 

i. Action plans established 

ii. Individuals assigned to tasks 

iii. Timelines established 

9. Overall, how useful/valuable do you think the 5E reports are in your school improvement planning? 

a. What data in the 5E reports are most helpful to you for your school improvement planning? What 
about the reports make the helpful? 

b. I understand that future reports will include the 5Essentials Scale Scores. The 5E scale scores will include 
(see list below) and will be based on a combination of different survey questions to develop a composite 
score. To what extent will the 5E scale scores be useful in your district/school? Will some of the scale 
scores be more useful than others? 

Effective Leaders 
Collaborative Teachers 
Involved Families 
Supportive Environments 
Ambitious Instruction 

c. In terms of all of the other priorities/activities occurring in your district/school, where do you see the 
5Essentails Survey data fitting in? 

i. What would make the 5E data more useful for your school improvement planning? 

ii. What could make the 5E data more meaningful? 

10. Has your district/school utilized the 5Essentails Survey reports for other purposes? 

a. LISTEN FOR OR ASK IF PROMPTS ARE NEEDED 

i. Influenced instruction or other practices 

ii. Promoted a positive school climate 

iii. Impacted collaboration among various stakeholders 

b. Can you describe one or two examples? 

1 1 . Have you shared your district’s (school’s) 5E Survey results with the School Board, parents or community? If yes, please 
explain. If not, please describe why not. 

12. What suggestions do you have to improve the 5E survey, administration of the survey or the results report to better assist 
in school improvement planning? 

IF YOU PLAN TO USE 5 Essentials data: 

13. How do you plan to use the 5Essentials data? 

a. School improvement planning (ask for examples) 

b. Other purposes (ask for examples) 

c. What data from the report will be helpful for your school improvement planning? 

d. I understand that future reports will include the 5Essentials Scale Scores. The 5E scale scores will include 
(see list below) and will be based on a combination of different survey questions to develop a composite 
score. To what extent will the 5E scale scores be useful in your district/school? Will some of the scale 
scores be more useful than others? 
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Effective Leaders 
Collaborative Teachers 
Involved Families 
Supportive Environments 
Ambitious Instruction 

e. In terms of all of the other priorities/activities occurring in your district/school, where do you see the 
5Essentails Survey data fitting in? 

i. What would make the 5E data more useful for your school improvement planning? 

ii. What could make the 5E data more meaningful? 

Do you plan to share your district’s (school’s) 5E Survey results with the School Board, parents, or community? If yes, 
please explain. If not, please describe why not. 

What suggestions do you have to improve the 5E survey, administration of the survey or the results report to better assist 
in school improvement planning? 

IF NOT USING SEssentials data: 

16. If you have not used the 5E results, could you explain why not? 

a. LISTEN FOR OR ASK IF PROMPTS ARE NEEDED 

i. Concerns about data validity, information shared with public, norming based on Chicago; use of data in 

evaluations, lack of time, delay in getting responses, small response rate, concerns about question wording, lack 
of connection to other school data systems 

b. What could be changed to encourage your district/school to use the 5E results? 

c. In terms of all of the other priorities/activities occurring in your district/school, where do you see the 
5Essentails Survey data fitting in? 

iii. What would make the 5E data more useful for your school improvement planning? 

iv. What could make the 5E data more meaningful? 

17. What suggestions do you have to improve the 5E survey, administration of the survey or the results report to better assist 
in school improvement planning? 

Supports for Use of Data 


14. 

15. 


For these next set of questions, I’m interested in hearing about supports for using data that may or may not be available in your 
district/school. I have different categories with 3-4 items each that describe different types of supports and I’ll ask you to indicate 
to what extent you think that the support is available at your school. And then I’ll ask for some examples of supports specifically 
about the 5Essentials data. (IF NO 5E data example, then ask for examples with other school data.) 

Data System Access 

18. Principals or specialists have access to all or most data for students in their school 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

19. Teachers have access to data on students in their classroom 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

20. OTHER examples of Data System Access with 5E data, or if not, then other data 
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Leadership for improvement and the use of data 

21. Training is available for school administrators/leaders on how to provide leadership for data-driven decision-making 
practices in their school 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

22. Leaders use data to identify professional development activities for schools identified for improvement 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

23. Leaders require all or particular schools to follow specific data-driving decision-making practices in their school 
improvement plans 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

24. Leaders follow up to determine if schools have implemented instructional changes prescribed as a result of data analysis 
activities 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

25. OTHER examples of Leadership with 5E data, or if not, then other data 

LISTEN FOR OR ASK IF PROMPTS ARE NEEDED 

i. Ensure necessary resources to promote this improvement 

ii. Model data use (e.g., use of data for evaluating programs, teachers, and principals) 

iii. Design and implement regular activities involving examining student data 

iv. Consider the ability to utilize data in decisions for hiring teachers 

v. Establish organizational climate of trust and mutual respect 

vi. Communicate expectations around data use; set expectations for staff buy-in and participation 

Tools for generating actionable data 

Online assessments are available in reading, mathematics, or other core subject areas 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

Library of diagnostic or benchmark assessments linked to academic standards is available (Web-accessible for 
downloading) 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

OTHER Examples ofTools to access to Student Data with 5E data, or if not, then other data 

LISTEN FOR OR ASK IF PROMPTS ARE NEEDED 

i. Aggregated data/ Student level data 

ii. Disaggregated into subgroups of interest 

iii. Formative assessments students take online 

iv. Timeliness of the data 

v. Static reports 

vi. Interactive data systems 

vii. Computer software to access data 

Social structures and time set aside for analyzing and interpreting data 

29. District/School implements policies and requirements to use data 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

30. District/School pays for incentives for teachers to use (or obtain training) in data-driving decision making (e.g., paying 
for dedicated time for schools staff to review data) 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 


26. 

27. 

28. 
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31. District/school requires “data conferences” between individual principals and their supervisors (FOR SCFIOOLS: 
principals and their teachers) 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

32. OTHER Examples of Social Structures and Time Set Aside with 5E data, or if not, then other data 

LISTEN FOR OR ASK IF PROMPTS ARE NEEDED 

i. Organize small groups within the school to review data 

ii. Promote collaborative discussions about data use and achievement 

iii. Set aside time for teachers to analyze and interpret data (drawing inferences from the data) and discuss in small 
groups (in general and specifically from the 5Essentials Survey) 

iv. Adoption of procedures for discussing data 

Professional development and technical support for data interpretation 

33. Training is available for principals, other administrators, and teachers on using the data system to analyze student 
achievement 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

34. Training is available for principals, other administrators, and teachers on using data to change instructional practice 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

35. Technical experts (in systems, networks, databases) are available to schools to support system use 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

36. Data analysis experts (e.g., data coaches) are available to school staff 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

37. OTHER examples of Professional Development or Technical Support with 5E data, or if not, then other data 

Tools for acting on data 

38. Models are provided to schools to illustrate how to use data in allocating resources and designing school improvement 
activities 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

39. Teachers are provided with processes or models of how to connect data to instructional and other practices 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

40. Teachers are provided with research-based guidance on differentiating instruction on the basis of student assessment 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

41. Database of lesson plans and planning resources (e.g., online) linked to academic standards and assessment results are 
provided. 

NOT AT ALL SOME EXTENT MODEST EXTENT GREAT EXTENT 

42. OTHER examples of Tools for Acting on Data with 5E data, if not, then other data 

LISTEN FOR OR ASK IF PROMPTS ARE NEEDED 

i. Identify or descriptions of best practices 

43. Is there anything else you would like to add or share with me about the 5E Survey and your district/school’s use of the 5E 
data? 
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Suggestions to Improve the Survey and its Administration 


O Market the survey — frame and motivate participation and utilization — in order to increase buy-in. 

• Respondents suggested that schools and districts need to be provided with a reason to use the 5Essentials survey and training 

to help them understand its empirical backing and potential benefits, particularly its relationship to school improvement. 

► “Again, it aligns really well with our strategic plan, therefore, I see it fitting in well, I see it as a priority for us if we can get the 
support and training to roll it out well to our principals, teachers, and families so that they understand, I can explain to them 
how it fits in, what I can’t explain to them is, the validity, reliability, and data analysis behind it. I need help with that. ” 

► “I would say market it from the stand point of trying to make successfid programs better because you don’t want to rest on your 
laurels and take things for granted. ” 

► “The residts and the benefits of the data. I think this data benefits this piece of student achievement or this data benefits this 
piece of professional development. So knowing the benefits of the survey would be — probably motivate us to use it. ” 

• Improve the 5Esentials website and make sure it is a “one-stop shop” for information and access to the survey and results. 

© Increase breadth of participation and response rates 

• Make sure promotional materials feature diversity, for example, with students who are demographically similar to those at a 

diverse array of schools. 

• Numerous strategies were reported for increasing participation: 

► “Oh they’ve stood in lines at pick-up handing things out. And hopefully once we’re able to present data to them, that makes them 
know their voice is valued and has been heard and matters. ” 

► “We significantly increased our response rate for our School Climate Surveys by having the parents fill out paper surveys that 
were Scantronned, that the kids took home, the parents filled them out, and the kids brought them back for a pizza party or 
something. . . . Our other model was we had the parents at parent-teacher conferences to go to the computer lab and take the 
survey. ...So it’s hard to get people to do the online surveys. ” 

► “One of the principals at our middle schools, she designated a week and was like, ‘Hey we’re going to bang this out. This is the 
week. This is our week. ”’ 

© Rostering 

• Don’t link names to responses - if not anonymous (with rostering) ensure respondents it is secure and confidential 

► “I believe when the survey first came out, teachers were still a little nervous about how confidential it would be. Despite probably 
numerous warnings that it’s going to be confidential, I don’t think people believed that it truly could or would be confidential. 
Any way that you could insure or kind of create some confidence in teachers that it’s confidential, that would be one thing. ’’ 

© Improving the survey items 

• Suggestions regarding specific items on the survey: 

► Perhaps further guidance/definition about the differences between “agree” vs. “strongly agree” and “disagree” vs. “strongly 
disagree”. 

► Some navigational issues, in terms of entering school names: some districts span multiple counties — clarify whether folks 
should select county of school or county where they live; spelling (Dupage vs. Du Page) 
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• Others suggested that perhaps some ambiguity surrounding the interpretation of responses could be alleviated by allowing 

providing space in the survey for user comments or clarifying narrative. 

► “Maybe if a teacher or a student could write a comment to follow up on a question, something like that might be usefid. 
Sometimes a narrative might help. ” 

• Other suggestions for making the survey more user-friendly included: 

► Adding option for paper surveys or smartphone apps in addition to current version 

► Link/ coordinate survey administration with administration of other required assessments. 

► “Could it be added in with ISAT or standardized testing so when you’re giving the standardized testing, here’s some touchy feely 
questions, so, it’s a little more palatable? The State says you have to give ISAT, oh, the State says you have to give the [5 Essentials] 
survey. Oh, you knoiv let’s save us some trouble and, and get the mandate done at the same time, the mandates done at the same time . ' 

© Timing of the survey 

• Allow a longer administration window and more flexibility as to when the survey is administered, particularly to coincide 

with when parents are already scheduled to be in the building. 

► “If we could have the parent timing at different times, to be able to do it on our — when we need to do it. I think that would be 
better. Like I said, when I get my parents in the building. . . ” 

Suggestions for Improving the Presentation and Usefulness of the Survey Results 


© Timing and Format of Results 

• Provide more immediate feedback, with results arriving preferably prior to the beginning of the next school year, to allow 

schools to address issues over summer break 

► “If you’re getting it back and you’re early summer and you’re able to get it into your improvement team, the Rising Star team 
that August meeting and say here’s what we saw and we can work it in into an early fall improvement. ” 

• Provide raw data allowing districts the ability to disaggregate responses within school. 

► Respondent 1: “You know, if it could be broken down to be in small, small groups to understand that data.” Respondent 2: “And 
even by department. ” Respondent 1: “Yeah, and that could be one way to break it down. It could be one person coming and 
talking to each department about the data, training on how to perhaps analyze it, break it down. ” 

• Disseminate results more widely, especially to teachers, and make access to results easier. 

► “Maybe make it more visible, cause I wasn’t even aware that that was on the school report card. . . .I’ve always wanted to 
know, like, what happened with it, but I didn’t really know like where I could look at all of it. So, maybe make it a little more 
accessible to all the teachers so they could see. ” 

► “OK, after I printed the junior high, I had to close out. I didn’t have an option to go back and get the high school. So I had to 
close out, I had to re-log on and every time I wanted to print something it didn’t give me the option to go back, it just kept going 
back to this document. ” 

► “Maybe send [the results] out to every participant. I know that we want to keep anonymity with it, however, then providing 
whatever email you might log into it woidd give that residts for the school district that you work in. Maybe then that would 
make it more accessible to the teachers and staff.” 

© Putting the data in context (reporting normed scores vs. frequency distributions) 

• Some respondents also suggested that their scores could be benchmarked against demographically similar districts or their 

own scores over time, rather than statewide (or CPS-based) norms. 
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► “Yeah, you know it would be more helpful, I think, than comparing us to a national norm or whatever. Just giving us our 
profile as a benchmark year one, then let’s look at year two and see, you know, let’s look for progress instead of, you know, where, 
how we compare to some other subset. ” 

© Public vs. private reporting 

• Several respondents suggested that the 5Essentials data should be for school improvement use only, and not reported publicly. 

► “So, my concern is that the 5Essentials will be presented as a stand-alone, and it will not paint the picture of actually what is 
happening at the school for parents. Then based on where we are in the world and the attitudes, that that could be detrimental 
to some schools where it doesn’t really need to be. So, I just think that how - 1 was so happy that they pulled back in some of 
the presentation of the data after the first one, because we really don’t know what it means with only one data point. I would 
hope that they would continue to review that very critically before presenting any data to the public. Because until you get three 
or four data points, you don’t really know what it means. And if you just put out a number, uneducated families are going to 
misinterpret it. For the good and for the [bad] — both ways. ” 

► “But even then, I don’t feel the State should compile that data. I think a school should be able to do it. I don’t think it should be 
placed on school report cards. I think that, you know, each school needs to be in control of that data. And, I don’t know, maybe 
we should report it to our constituents, but I don’t think that we need to be rated and ranked and compared to other schools. I 
think we should be in charge of our own survey and the questions on it. I just don’t think that the State needs to play a role in 
this. I think it should be a local thing. I’m going to compare it to students and academics, I think it should be a growth kind of 
thing where we decide what the questions are and then we use the same questionnaire or survey the next year to see how we grow 
in terms of our climate. ” 

©Provide more actionable feedback 

• Respondents suggested that more training in data analysis and utilization was required to help understand and use the survey 

results. 

► “We know we should be using the data. We know we should know how to read the data, but no one seems to know how to teach 
us how to do that. And whether it’s personal to a district or personal to a building or specific to a building or even in general, no 
one really seems to know what to do. ” 

► “Right, and I don’t think a lot of people have had training on just how to read data, how to interpret what we’re getting back in 
the classrooms and how to make it applicable in our plan. ” 

► “Yeah, and it is, again it’s the professional development and in fact, ...we need to have the people that presented at the regional 
networking meeting to come into our district and spend an afternoon with our principals doing that exact same presentation 
and running through with our data and that might be all that we need. ” 

• Respondents wanted assistance in using the 5Es survey results to easily identify problems or emerging issues at their schools. 

► “If through the data it came back ranked based on all our results. This is your weakest area, this is your strongest area. Then we 
could go from there. ” 

► “I think some help with interpreting 5Essentials data as identifying based on what we are seeing that these are the one or two or 
three likely priorities for school improvement, that would probably be very helpful. ” 

• Next, they would like more concrete advice for “next steps” about how to improve weaknesses identified by the 5Essentials 

survey results. 

► “And then probably, you know, two parts: understanding the data or at least have somebody break it down. You know, I came 
up with the raw numbers to see what was going on, but I wouldn’t expect, you know, everybody in the building to be able to do 
that. But then to then look at what are the next steps after that. The data is showing we’re weak in a certain, what, what next?” 
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Appendix C: Survey Tables for ISBE Implementation Feedback Survey 


Table C-1 . Teacher Survey - Superintendents and Principals by Region 


Region 

Generated 

Valuable 

Data 

Administration 

Communication 

Sufficient 

Teachers 
Willing to 
Participate 

Data Were 
Fair and 
Reliable 

Items 
Addressed 
Issues 
of Local 
Interest 

Logistical 
Challenges 
Due to 

Technological 

Issues 

Challenges 
for Other 
Reasons 



% 

27% 

58% 

62% 

31% 

20% 

31% 

39% 

Cook 


n 

26 

26 

26 

26 

25 

26 

26 


% 

59% 

81% 

81% 

62% 

52% 

12% 

16% 



n 

94 

94 

94 

93 

93 

93 

93 



% 

38% 

68% 

76% 

53% 

37% 

n% 

21% 

Suburban Chicago 


n 

37 

38 

38 

38 

38 

38 

38 

Principal 

% 

48% 

74% 

78% 

55% 

47% 

6% 

15% 


n 

118 

118 

118 

118 

117 

117 

118 



% 

28% 

59% 

60% 

26% 

33% 

21% 

29% 

Northern Illinois 


n 

53 

53 

53 

53 

52 

53 

51 

Principal 

% 

44% 

66% 

80% 

59% 

50% 

15% 

18% 


n 

89 

90 

89 

90 

88 

89 

88 



% 

33% 

51% 

58% 

26% 

36% 

17% 

35% 

Central Illinois 


n 

77 

77 

77 

77 

76 

77 

77 

Principal 

% 

48% 

70% 

81% 

48% 

46% 

14% 

25% 


n 

154 

155 

155 

155 

154 

155 

154 



% 

48% 

64% 

77% 

41% 

59% 

n% 

30% 

Southern Illinois 


n 

44 

44 

44 

44 

44 

44 

44 


% 

40% 

58% 

78% 

36% 

36% 

15% 

18% 



n 

55 

55 

54 

55 

55 

55 

55 


Table C-2. Teacher Survey - Superintendents and Principals by Size 


District/School Size 

Generated 

Valuable 

Data 

Administration 

Communication 

Sufficient 

Teachers 
Willing to 
Participate 

Data Were 
Fair and 
Reliable 

Items 
Addressed 
Issues 
of Local 
Interest 

Logistical 
Challenges 
Due to 

Technological 

Issues 

Challenges 
for Other 
Reasons 



% 

41% 

53% 

72% 

42% 

44% 

17% 

27% 

<600 

Super ii iler i(Jenl 

n 

64 

64 

64 

64 

63 

64 

64 

Principal 

% 

44% 

70% 

78% 

51% 

53% 

18% 

23% 


n 

90 

91 

90 

91 

89 

91 

90 



% 

33% 

58% 

63% 

26% 

36% 

16% 

32% 

600-2,000 


n 

120 

120 

120 

120 

118 

120 

119 

Principal 

% 

40% 

64% 

80% 

46% 

35% 

7% 

22% 


n 

190 

191 

190 

191 

190 

188 

189 



% 

31% 

64% 

64% 

40% 

32% 

21% 

35% 

> 2,000 


n 

52 

53 

53 

53 

53 

53 

52 


% 

57% 

76% 

80% 

59% 

55% 

14% 

15% 



n 

230 

230 

230 

229 

228 

230 

229 
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Table C-3. Student Survey - Superintendents and Principals by Region 


Region 

Generated 

Valuable 

Data 

Administration 

Communication 

Sufficient 

Students 
Willing to 
Participate 

Data Were 
Fair and 
Reliable 

Items 

Addressed 
Issues 
of Local 
Interest 

Logistical 
Challenges 
Due to 

Technological 

Issues 

Challenges 
for Other 
Reasons 



% 

42% 

65% 

81% 

48% 

46% 

42% 

39% 

Cook 


n 

26 

26 

26 

25 

26 

26 

26 


% 

69% 

80% 

82% 

65% 

57% 

19% 

22% 



n 

75 

74 

74 

74 

74 

73 

74 



% 

54% 

89% 

76% 

54% 

38% 

8% 

n% 

Suburban Chicago 


n 

37 

37 

37 

37 

37 

37 

37 


% 

47% 

73% 

81% 

54% 

49% 

16% 

18% 



n 

74 

74 

73 

74 

74 

73 

73 



% 

44% 

67% 

65% 

44% 

35% 

19% 

17% 

Northern Illinois 


n 

52 

52 

52 

52 

51 

52 

52 


% 

49% 

72% 

62% 

59% 

43% 

27% 

28% 



n 

68 

68 

68 

68 

68 

67 

67 



% 

33% 

53% 

63% 

34% 

45% 

22% 

29% 

Central Illinois 


n 

77 

77 

75 

77 

76 

77 

77 


% 

43% 

67% 

70% 

50% 

40% 

n% 

22% 



n 

122 

121 

122 

121 

121 

122 

121 



% 

41% 

61% 

64% 

43% 

43% 

25% 

34% 

Southern Illinois 


n 

44 

44 

44 

44 

44 

44 

44 


% 

46% 

76% 

80% 

50% 

35% 

26% 

22% 



n 

46 

46 

45 

46 

46 

46 

45 


Table C-4. Student Survey - Superintendents and Principals by Size 


District/School Size 

Generated 

Valuable 

Data 

Administration 

Communication 

Sufficient 

Students 
Willing to 
Participate 

Data Were 
Fair and 
Reliable 

Items 

Addressed 
Issues 
of Local 
Interest 

Logistical 
Challenges 
Due to 

Technological 

Issues 

Challenges 
for Other 
Reasons 



% 

41% 

58% 

73% 

44% 

47% 

20% 

23% 

<600 


n 

64 

64 

64 

63 

64 

64 

64 


% 

47% 

81% 

83% 

59% 

45% 

15% 

23% 



n 

83 

83 

83 

82 

83 

83 

82 



% 

38% 

63% 

63% 

39% 

37% 

24% 

28% 

600-2,000 


n 

119 

119 

118 

119 

117 

119 

119 


% 

44% 

69% 

73% 

45% 

35% 

15% 

23% 



n 

144 

142 

142 

143 

143 

142 

142 



% 

48% 

77% 

71% 

48% 

44% 

19% 

23% 

> 2,000 


n 

52 

52 

51 

52 

52 

52 

52 


% 

58% 

72% 

71% 

63% 

54% 

23% 

21% 



n 

158 

158 

157 

158 

157 

156 

156 


www.siue.edu/ierc 59 


Use of the Illinois 5Essentials Survey Data 


Use of the Illinois 5Essentials Survey Data 



Table C-5. Report - Superintendents and Principals by Region 


Region 

Valuable 
Information 
for Role 

New 

Information 
for Role 

Easy to Use 
Information 
for Role 

Teachers 

Found 

Reports 

Valuable 

Parents 

Found 

Reports 

Valuable 



% 

27% 

31% 

36% 

23% 

15% 

Cook 


n 

26 

26 

25 

26 

26 

Principal 

% 

54% 

44% 

53% 

39% 

38% 


n 

94 

94 

94 

89 

89 



% 

23% 

23% 

28% 

18% 

8% 

Suburban Chicago 


n 

39 

39 

39 

39 

38 

Principal 

% 

40% 

38% 

41% 

23% 

21% 


n 

116 

116 

116 

115 

106 



% 

25% 

17% 

26% 

15% 

10% 

Northern Illinois 


n 

53 

53 

53 

53 

52 

Principal 

% 

41% 

37% 

32% 

28% 

16% 


n 

86 

89 

89 

89 

80 



% 

18% 

21% 

25% 

14% 

9% 

Central Illinois 


n 

77 

77 

77 

77 

77 

Principal 

% 

39% 

38% 

40% 

26% 

19% 


n 

156 

156 

156 

153 

146 



% 

32% 

43% 

34% 

16% 

5% 

Southern Illinois 


n 

44 

44 

44 

44 

42 

Principal 

% 

34% 

39% 

44% 

26% 

21% 


n 

56 

56 

55 

54 

53 


Table C-6. Report - Superintendents and Principals by Size 


District/School Size 

Valuable 
Information 
for Role 

New 

Information 
for Role 

Easy to Use 
Information 
for Role 

Teachers 

Found 

Reports 

Valuable 

Parents 

Found 

Reports 

Valuable 



% 

27% 

28% 

32% 

13% 

n% 

<600 


n 

64 

64 

63 

64 

63 


% 

36% 

38% 

43% 

23% 

22% 



n 

90 

90 

89 

90 

86 



% 

21% 

22% 

25% 

17% 

7% 

600-2,000 


n 

121 

121 

121 

121 

119 

Principal 

% 

33% 

32% 

33% 

20% 

16% 


n 

189 

192 

192 

189 

180 



% 

26% 

28% 

32% 

21% 

12% 

> 2,000 


n 

53 

53 

53 

53 

52 

Principal 

% 

52% 

44% 

48% 

36% 

28% 


n 

229 

229 

229 

221 

208 
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Table C-7. Use of 5E Data for Planning - Superintendents and Principals by Region 


Region 

Planning and / 
or Continuous 
Improvement 

Modified 

District/School 

Improvement 

Plan 

Used Data 
Other 
Reasons 



% 

31% 

23% 

35% 

Cook 


n 

26 

26 

26 

Principal 

% 

60% 

40% 

54% 


n 

93 

93 

91 


Superintendent 

% 

31% 

10% 

28% 

Suburban Chicago 

n 

39 

39 

39 

Principal 

% 

44% 

22% 

35% 


n 

117 

116 

113 



% 

32% 

19% 

19% 

Northern Illinois 


n 

53 

53 

53 

Principal 

% 

41% 

24% 

36% 


n 

90 

89 

89 



% 

26% 

8% 

25% 

Central Illinois 


n 

76 

76 

75 

Principal 

% 

47% 

28% 

39% 


n 

156 

155 

152 



% 

34% 

9% 

19% 

Southern Illinois 


n 

44 

44 

42 

Principal 

% 

43% 

18% 

40% 


n 

56 

56 

55 


Table C-8. Use of 5E Data for Planning - Superintendents and Principals by Size 


District/School Size 

Planning and / 
or Continuous 
Improvement 

Modified 

District/School 

Improvement 

Plan 

Used Data 
Other 
Reasons 


Superintendent 

% 

27% 

n% 

23% 

< 600 

n 

63 

63 

60 

Principal 

% 

42% 

20% 

40% 


n 

91 

89 

90 


Superintendent 

% 

29% 

11% 

20% 

600-2,000 

n 

121 

121 

121 

Principal 

% 

43% 

25% 

32% 


n 

192 

192 

187 


Superintendent 

% 

36% 

17% 

34% 

> 2,000 

n 

53 

53 

53 


% 

54% 

32% 

47% 



n 

229 

228 

224 
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Contact the IERC toll-free at 1-866-799-IERC (4372) 
or by email at ierc@siue.edu. 
http://www.siue.edu/ierc 


The Illinois Education Research Council at Southern Illinois University Edwardsville was 
established in 2000 to provide Illinois with education research to support P-20 education 
policy making and program development. The IERC undertakes independent research and 
policy analysis, often in collaboration with other researchers, that informs and strengthens 
Illinois’ commitment to providing a seamless system of educational opportunities for its 
citizens. Through publications, presentations, participation on committees, and a research 
symposium, the IERC brings objective and reliable 
evidence to the work of state policymakers and practitioners. 



ILLINOIS EDUCATION RESEARCH COUNCIL 


